Intelligent vehicle checking auxiliary system based on real-time monitoring video analysis

Abstract

The present application relates to the field of railway transportation maintenance, and more particularly to a smart vehicle inspection auxiliary system based on real-time monitoring video analysis, comprising: a work evaluation module configured to determine a risk analysis terminal, determine whether to adjust a work coordination process to a work risk analysis based on a risk target proportion index and a risk related execution index; a risk analysis module configured to perform work risk analysis on each risk analysis terminal, and determine the category of the corresponding risk analysis terminal based on a stage target intervention index and a stage target intersection index; an optimized execution module configured to determine whether the process of performing work frame compensation processing on the corresponding risk analysis terminal includes work target compensation processing based on the category of each risk analysis terminal; and a coordination execution module configured to perform work coordination processing on each risk analysis terminal, which improves the supervision quality of the inspection and repair process of railway freight vehicles while ensuring the utilization efficiency of transmission resources.

Description

Technical Field

[0001] This invention relates to the field of railway transportation maintenance, and in particular to an intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis. Background Technology

[0002] In the process of inspecting and maintaining railway freight cars, to ensure the quality of maintenance work and the safety of maintenance personnel, each maintenance worker is equipped with a video recording terminal for real-time video monitoring of the operation. Each video recording terminal needs to upload the acquired video information to a central node to determine if any abnormalities exist during the inspection process. However, continuously uploading the acquired video information in its entirety can lead to significant waste of transmission resources during the operation. Therefore, frame extraction processing is necessary to ensure the effective utilization of transmission resources during the actual operation. However, existing technologies often use fixed frame extraction schemes, which cannot effectively balance monitoring quality and transmission efficiency. Therefore, how to optimize the frame extraction process in real time based on the actual operation process of the maintenance personnel corresponding to each video recording terminal, so as to reduce the consumption of transmission resources while ensuring the monitoring quality of maintenance work, is a problem that urgently needs to be solved by those skilled in the art.

[0003] Chinese patent application publication number CN120599514A discloses a video detection system for railway freight car component replacement operations, solving the problem of how to improve the efficiency of railway freight car maintenance operation detection, belonging to the field of railway freight car maintenance technology. The application includes: a video processing layer that processes real-time acquired video streams of depot operations to obtain dynamic frame images; a multi-target detection and tracking module that detects the dynamic frame images, obtains the target's status label, and determines the target's first appearance time, last disappearance time, and position coordinates, forming spatiotemporal sequence data; an action timing analysis module that analyzes the target's status label and spatiotemporal sequence data, filters out abnormal action nodes, and extracts the action nodes and corresponding times of state transition scenarios; a component status judgment module that verifies the final placement status of the component and the compliance of the operation process based on the target's status label and spatiotemporal sequence data; and a decision output layer that generates a structured detection report based on the analysis results. However, the above solution has the following shortcomings: it fails to adjust and optimize the frame extraction process in a timely and effective manner based on the personnel's work situation and operational needs corresponding to each video acquisition terminal, resulting in the acquired video frame information failing to meet the requirements for data transmission resources while ensuring the quality of supervision of the maintenance operation process. Summary of the Invention

[0004] To address this issue, the present invention provides an intelligent vehicle inspection assistance system based on real-time monitoring video analysis. This system overcomes the problem that existing technologies fail to adjust and optimize the frame extraction process in a timely and effective manner by considering the personnel's work situation and operational needs at each video acquisition terminal. Consequently, the acquired video frame information cannot effectively balance the monitoring quality of the maintenance operation process with the data transmission resource requirements.

[0005] To achieve the above objectives, the present invention provides an intelligent vehicle inspection assistance system based on real-time monitoring video analysis, comprising:

[0006] The operation assessment module is used to conduct operation risk assessment for the vehicle inspection operation process, in order to determine the risk analysis terminal. Based on the risk target proportion index and the risk-related execution index, it determines whether to adjust the operation assistance process to operation risk analysis. The risk analysis terminal is determined according to the operation risk coefficient of each operation monitoring terminal.

[0007] The risk analysis module, which is connected to the operation evaluation module, is used to perform operation risk analysis for each risk analysis terminal and determine the category of the corresponding risk analysis terminal based on the stage target interference index and stage target cross index of each risk analysis terminal.

[0008] An optimized execution module, which is connected to the risk analysis module, is used to determine whether the process of performing work frame extraction compensation processing for the corresponding risk analysis terminal includes work target compensation processing based on the category of each risk analysis terminal, to determine the frame extraction execution parameters based on the maintenance work interference index of the compensation target set, and to determine the regulatory compensation terminal for each maintenance execution target of the risk analysis terminal based on the work-related index.

[0009] The assistant execution module, which is connected to the job evaluation module, is used to perform job assistant processing for each risk analysis terminal to determine the assistant job terminal and frame extraction execution parameters for each risk analysis terminal.

[0010] Furthermore, the job evaluation module periodically detects the job risk coefficient of each job monitoring terminal;

[0011] The risk analysis terminal is a job monitoring terminal whose job risk coefficient is greater than the preset job risk coefficient;

[0012] The operational risk coefficient is determined based on the abnormal execution index of the personnel corresponding to each operational monitoring terminal and the stage target interference index.

[0013] Furthermore, the risk analysis module responds to the analysis execution conditions, performs operational risk analysis on each risk analysis terminal, and determines the operational category of each risk analysis terminal;

[0014] The analysis execution condition is that the risk target percentage index determined in the analysis execution cycle is greater than the preset risk target percentage index or the risk-related execution index is greater than the preset risk-related execution index.

[0015] Furthermore, the risk analysis terminals are categorized into Class I risk analysis terminals and Class II risk analysis terminals;

[0016] The first type of risk analysis terminal is a risk analysis terminal in which the stage target interference index is greater than the preset stage target interference index or the stage target cross index is greater than the preset stage cross index.

[0017] The second type of risk analysis terminal is a risk analysis terminal where the stage target interference index is less than or equal to the preset stage target interference index and the stage target cross index is less than or equal to the preset stage cross index.

[0018] Furthermore, the optimization execution module performs work target compensation processing for a type of risk analysis terminal, and determines the frame extraction execution parameters of the corresponding compensation target set based on the maintenance operation interference index of each compensation target set of the type of risk analysis terminal.

[0019] The frame extraction execution parameters are positively correlated with the maintenance operation interference index of the compensation target set.

[0020] Furthermore, the maintenance operation interference index of any set of compensation targets is determined based on the execution interference parameters of each maintenance execution target within the set of compensation targets;

[0021] The set of compensation targets is determined based on interference-related compensation parameters.

[0022] Furthermore, the optimization execution module performs work frame extraction compensation processing on the second-class risk analysis terminal or the first-class risk analysis terminal that has completed the work target compensation processing, and determines the supervision compensation terminal for each maintenance execution target of the work compensation terminal based on the work-related index.

[0023] The job compensation terminal is a type II risk analysis terminal and a type I risk analysis terminal that completes job target compensation processing.

[0024] Furthermore, the optimization execution module determines the relevant monitoring terminal for each job compensation terminal based on the execution timing correlation index and the execution space correlation index;

[0025] The execution time correlation index of any of the relevant monitoring terminals to the corresponding job compensation terminals is greater than the preset execution time correlation index, and the execution space correlation index is greater than the preset execution space correlation index.

[0026] Furthermore, the coordinating execution module responds to the coordinating execution conditions, performs job coordinating processing for each risk analysis terminal, and determines the coordinating job terminal for each risk analysis terminal based on the execution target overlap index and the execution target cross index.

[0027] The coordinating execution conditions are that the risk target percentage index determined by the analysis execution cycle is less than or equal to the preset risk target percentage index, and the risk-related execution index is less than or equal to the preset risk-related execution index.

[0028] Furthermore, the frame-skipping execution parameters for each co-executor combination are determined based on the cross-index of the execution target;

[0029] Any of the aforementioned assistant execution combinations is a combination of any risk analysis terminal and its corresponding assistant operation terminal;

[0030] The frame extraction execution parameters of each coordinator execution group are positively correlated with the execution target cross-index between the risk analysis terminal and the coordinator operation terminal within the corresponding coordinator execution group.

[0031] Compared with the prior art, the beneficial effects of the present invention are that the technical solution of the present invention periodically analyzes the work status of maintenance personnel corresponding to each work monitoring terminal and whether there are normal abnormal risks in the supervision process of subsequent maintenance tasks. In this way, it determines whether the maintenance quality and supervision quality of each work monitoring terminal are estimated within a certain time range, thereby identifying the risk analysis terminals that need further analysis. Combined with the risk target proportion index and risk-related execution index in the actual operation process, it determines targeted optimization strategies to ensure that the maintenance supervision process of each risk analysis target is optimized in a targeted manner. While ensuring the efficient utilization of video frame information transmission resources, it improves the supervision quality of the maintenance operation process.

[0032] Furthermore, this invention determines whether to adjust the operation assistance process to operation risk analysis based on the risk target proportion index and the risk-related execution index. Combining the overall status of the risk analysis terminals in the actual operation process, a targeted analysis and processing method is determined. The risk target proportion index and the risk-related execution index characterize the richness of the existing risk analysis terminals and the correlation of the operation processes of the risk analysis terminals. This makes the optimization scheme determined for the supervision process of each risk analysis terminal more consistent with the actual maintenance operation situation, thereby improving the optimization quality and processing efficiency of the optimization scheme. This invention improves the effectiveness of the optimization results for the maintenance supervision process.

[0033] Furthermore, in this invention, when the risk target proportion index is large or the risk-related execution index is large, operational risk analysis is performed on each risk analysis terminal. Since there are many risk analysis terminals or the operational processes of the risk analysis terminals are closely related, coordinating operational processes can easily have a significant impact on the deployment and execution of the actual maintenance operation. By performing operational risk analysis, the supervision plan for the operational process of the actual maintenance execution target is optimized in a targeted manner. This invention ensures the quality of actual supervision of the maintenance operation process without affecting the efficiency of normal maintenance operations.

[0034] Furthermore, this invention performs operational risk analysis on each risk analysis terminal and optimizes the monitoring process of each terminal based on its category. Specifically, the category of each risk analysis terminal is determined by the stage target interference index and the stage target cross-index. This indicates whether the dominant factors of the normalized abnormal risks in the monitoring quality of each terminal are interfered with by the inherent location of the maintenance operation. Targeted adjustments are then made accordingly. By effectively classifying the existing risk analysis terminals, unnecessary resource consumption and maintenance operation interference caused by uniform operational target compensation processing are avoided. In addition, for Category II risk analysis terminals or Category I risk analysis terminals that have completed operational target compensation processing, frame extraction compensation processing is performed to further ensure the monitoring quality of the maintenance operation process. This invention improves the monitoring quality of the maintenance operation process while ensuring the efficient utilization of data transmission and data analysis resources. Attached Figure Description

[0035] Figure 1 This is a module connection diagram of the intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis of the present invention;

[0036] Figure 2 This is a flowchart illustrating the present invention's method for determining whether to adjust task assistance processing to task risk analysis based on risk target proportion index and risk-related execution index.

[0037] Figure 3 This is a flowchart illustrating how the present invention determines the category of a risk analysis terminal based on the stage target interference index and stage target cross index of each risk analysis terminal.

[0038] Figure 4 This invention provides a flowchart illustrating whether the process of determining, based on the category of each risk analysis terminal, for performing job frame extraction compensation processing on the corresponding risk analysis terminal includes job target compensation processing. Detailed Implementation

[0039] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0040] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0041] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0042] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0043] Please see Figures 1 to 4 As shown, the present invention provides an intelligent vehicle inspection assistance system based on real-time monitoring video analysis, comprising:

[0044] The operation assessment module is used to conduct operation risk assessment for the vehicle inspection operation process, in order to determine the risk analysis terminal. Based on the risk target proportion index and the risk-related execution index, it determines whether to adjust the operation assistance process to operation risk analysis. The risk analysis terminal is determined according to the operation risk coefficient of each operation monitoring terminal.

[0045] The risk analysis module, which is connected to the operation evaluation module, is used to perform operation risk analysis for each risk analysis terminal and determine the category of the corresponding risk analysis terminal based on the stage target interference index and stage target cross index of each risk analysis terminal.

[0046] An optimized execution module, which is connected to the risk analysis module, is used to determine whether the process of performing work frame extraction compensation processing for the corresponding risk analysis terminal includes work target compensation processing based on the category of each risk analysis terminal, to determine the frame extraction execution parameters based on the maintenance work interference index of the compensation target set, and to determine the regulatory compensation terminal for each maintenance execution target of the risk analysis terminal based on the work-related index.

[0047] The assistant execution module, which is connected to the job evaluation module, is used to perform job assistant processing for each risk analysis terminal to determine the assistant job terminal and frame extraction execution parameters for each risk analysis terminal.

[0048] This invention optimizes the frame extraction scheme for video information acquired by video acquisition terminals during interval operations, ensuring the quality of video frame information for monitoring and transmission efficiency of maintenance work. The current maintenance operation being monitored is designated as the target monitoring process. This invention involves several maintenance personnel performing maintenance on railway freight cars. Each maintenance personnel is equipped with a corresponding work monitoring terminal. The work monitoring terminal performs real-time video monitoring of the corresponding maintenance personnel's maintenance process. Each work monitoring terminal performs frame extraction processing on the acquired monitoring video, transmits the acquired video frames in real-time, and stores the complete video locally. This invention assigns different maintenance locations to each maintenance personnel within different time ranges, designated as the corresponding maintenance execution target. This invention does not specify the specific model or type of the work monitoring terminal, but it must ensure real-time acquisition and frame extraction of video monitoring data. Each work monitoring terminal is equipped with a vibration sensor to obtain the corresponding vibration amplitude.

[0049] In this invention, each work monitoring terminal is configured with maintenance execution targets for each work evaluation cycle. The maintenance sequence is set based on the spatial distance between the specific locations of these targets, avoiding unnecessary consumption of personnel resources. For a single work monitoring terminal, the estimated execution time for any maintenance execution target is set according to the determined maintenance sequence and execution duration. For the i-th maintenance execution target within the corresponding evaluation cycle, the estimated execution time... , This marks the start time of the evaluation execution cycle. To evaluate the maintenance operation duration of the j-th maintenance execution target within the execution cycle, for a single maintenance execution target, the maintenance operation duration is the sum of the standard maintenance duration required for that maintenance execution target and the time required for maintenance personnel to travel from that maintenance execution target to the next maintenance execution target. In this invention, a fixed frame extraction execution parameter is used to obtain video frames of the monitoring video acquired by the corresponding conventional operation terminal. For a single conventional operation terminal, frame extraction processing is performed on the monitoring video acquired during the maintenance operation based on the frame extraction execution parameter initially set by the user. The frame extraction execution parameter is the number of video frames extracted per second. One possible value for the frame extraction execution parameter initially set by the user is 15fps.

[0050] Specifically, the job evaluation module periodically detects the job risk coefficient of each job monitoring terminal;

[0051] The risk analysis terminal is a job monitoring terminal whose job risk coefficient is greater than the preset job risk coefficient;

[0052] The operational risk coefficient is determined based on the abnormal execution index of the personnel corresponding to each operational monitoring terminal and the stage target interference index.

[0053] In this invention, the operational risk coefficient of each existing operational monitoring terminal is analyzed periodically. Specifically, an evaluation execution cycle is set, and the operational risk coefficient of each operational monitoring terminal is determined once in each evaluation execution cycle. A value for the evaluation execution cycle is provided, which is 20 minutes. For a single operational monitoring terminal, the operational risk coefficient is the sum of the products of the abnormal execution index and the stage target interference index of that operational monitoring terminal, respectively, and their corresponding evaluation weight coefficients. The abnormal execution index... , This represents the average vibration amplitude of the monitoring terminal during each instance of abnormal vibration within the evaluation execution cycle. For a preset vibration amplitude, if the vibration amplitude detected by the work monitoring terminal during the evaluation execution cycle is greater than the preset vibration amplitude, it is recorded as an abnormal vibration. The stage target interference index is the average value of the execution interference parameters of each maintenance execution target existing in the next evaluation execution cycle of the current evaluation execution cycle for work risk coefficient analysis. For a single maintenance execution target, the execution interference parameter is the average value of the work occlusion degree of each video frame image obtained for that maintenance execution target. For a single video frame image corresponding to that maintenance execution target, the work occlusion degree... , This refers to the number of pixels within the pixel range corresponding to the maintenance target in the video frame image. This refers to the number of pixels within the theoretical pixel range corresponding to the maintenance target in the video frame image. The theoretical pixel range is calculated based on pixels and the physical size of the maintenance target. This provides a method for determining the number of pixels within the theoretical pixel range by acquiring any unobstructed image of the maintenance target, along with the corresponding focal length value and camera shooting distance at the time of image acquisition. , To obtain the camera shooting distance used to determine the theoretical pixel range, To obtain the focal length value used to determine the theoretical pixel range, The longer side dimension of the visible surface of the target area for this maintenance operation. The visible short side dimension of the target area for this maintenance operation;

[0054] The conventional operation terminal is an operation monitoring terminal with an operation risk coefficient less than or equal to a preset operation risk coefficient. The evaluation weight coefficients corresponding to the abnormal execution index and the stage target interference index can be set by the user according to actual working conditions. One possible value for the evaluation weight coefficients is 0.4 for the abnormal execution index and 0.6 for the stage target interference index. For the preset operation risk coefficient and preset vibration amplitude, those skilled in the art will readily understand that the higher the user's requirements for the quality of supervision of the maintenance personnel's maintenance execution process, the larger the preset operation risk coefficient and the smaller the preset vibration amplitude. The operation risk coefficient characterizes the quality of maintenance execution by the maintenance personnel corresponding to the operation monitoring terminal within the corresponding evaluation execution cycle and the difficulty of monitoring the execution process of the maintenance execution target in subsequent evaluation execution cycles. One possible value for the preset operation risk coefficient is 1.1, and one possible value for the preset vibration amplitude is 4.5 m / s. 2 .

[0055] Specifically, the risk analysis module responds to the analysis execution conditions and performs operational risk analysis on each risk analysis terminal to determine the operational category of each risk analysis terminal;

[0056] The analysis execution condition is that the risk target percentage index determined in the analysis execution cycle is greater than the preset risk target percentage index or the risk-related execution index is greater than the preset risk-related execution index.

[0057] The current assessment and execution cycle for determining the operational risk coefficients of each operational monitoring terminal is recorded as the analysis execution cycle, and the risk target percentage index is... , The number of risk analysis terminals determined by the analysis execution cycle, The target monitoring terminal is the number of operational monitoring terminals present during the target supervision process. The risk-related execution index is the average of the execution target relevance indices of each risk analysis terminal. For a single risk analysis terminal, the execution target relevance index is... , This refers to the number of relevant execution objectives that exist within the next evaluation execution cycle of the risk analysis terminal during the analysis execution cycle. The number of maintenance execution targets existing in the next evaluation execution cycle of the analysis execution cycle of the risk analysis terminal, wherein the relevant execution targets are maintenance execution targets that exist together with other risk analysis terminals;

[0058] The values ​​of the preset risk target proportion index and the preset risk-related execution index are understood to reflect the higher the user's requirements for the transmission efficiency and effectiveness of the video frames monitoring the maintenance execution process of the maintenance personnel. The higher these requirements, the lower the preset operation risk coefficient and the lower the preset risk-related execution index. The risk target proportion index represents the proportion of existing risk analysis terminals among the actual operation monitoring terminals. The execution target correlation index represents the degree of correlation between the maintenance operation targets corresponding to each existing risk analysis terminal. When the determined risk target proportion index is greater than the preset risk target proportion index or the risk-related execution index is greater than the preset risk-related execution index, due to the abundance of existing risk analysis terminals or the significant correlation between the operation processes of these terminals, it is impossible to ensure the quality of actual supervision of the maintenance operation process without affecting the efficiency of normal maintenance operations through collaborative operation processing. Therefore, operation risk analysis is performed to optimize the video frame extraction strategy of each risk analysis terminal, providing a preset risk target proportion index and a preset risk-related execution index value of 0.35 and 0.2.

[0059] Specifically, the risk analysis terminals are categorized into Class I risk analysis terminals and Class II risk analysis terminals;

[0060] The first type of risk analysis terminal is a risk analysis terminal in which the stage target interference index is greater than the preset stage target interference index or the stage target cross index is greater than the preset stage target cross index.

[0061] The second type of risk analysis terminal is a risk analysis terminal where the stage target interference index is less than or equal to the preset stage target interference index and the stage target cross index is less than or equal to the preset stage target cross index.

[0062] Specifically, for a single risk analysis terminal, the stage target cross index is the average value of the workspace cross parameters of the maintenance execution targets existing in the next evaluation execution cycle of the analysis execution cycle of the risk analysis terminal. For a single maintenance execution target, the workspace cross parameter is the number of maintenance execution targets existing in the cross evaluation space of the maintenance execution target. The cross evaluation space is a spherical space with the centroid of the maintenance execution target as the center and the preset evaluation distance as the radius of the sphere.

[0063] The values ​​of the preset stage target interference index, preset stage target cross index, and preset evaluation distance are understood to reflect the following: the higher the user's requirements for the quality of supervision of the maintenance execution process by maintenance personnel, the smaller the value of the preset stage target interference index and the preset stage target cross index, and the larger the value of the preset evaluation distance. The stage target interference index represents the degree to which the risk analysis terminal is susceptible to affecting the overall quality of the maintenance images acquired during the operation process of the maintenance execution target in the next evaluation execution cycle of the analysis execution cycle. The stage target cross index represents the magnitude of the spatial cross-occlusion risk of the maintenance execution process of the maintenance execution target in the next evaluation execution cycle of the analysis execution cycle. This study investigates the degree of quality risk in image acquisition during the maintenance execution target image monitoring process of risk analysis terminals when the stage target interference index is greater than the preset stage target interference index or the stage target cross index is greater than the preset stage target cross index. This characterizes the dominant factors in the video monitoring process of the risk analysis terminal during maintenance operations. Based on this, risk analysis terminals are classified, and the frame extraction process of video information from each risk analysis terminal is optimized. A preset stage target interference index and a preset stage target cross index are provided, with the preset stage target interference index set to 0.3, the preset stage target cross index set to 3, and the preset evaluation distance set to 0.8m.

[0064] Specifically, the optimization execution module performs work target compensation processing for a type of risk analysis terminal, and determines the frame extraction execution parameters of the corresponding compensation target set based on the maintenance operation interference index of each compensation target set of the type of risk analysis terminal.

[0065] The frame extraction execution parameters are positively correlated with the maintenance operation interference index of the compensation target set.

[0066] Specifically, the maintenance operation interference index of any set of compensation targets is determined based on the execution interference parameters of each maintenance execution target within the set of compensation targets;

[0067] The set of compensation targets is determined based on interference-related compensation parameters.

[0068] Among them, since the stage target interference index or stage target cross-index of a certain type of risk analysis terminal is large, it indicates that the overall quality of the maintenance images acquired during the operation of the maintenance execution target is easily affected, or the maintenance process of the maintenance execution target has a large degree of spatial cross-interference. This indicates that the inherent position of the maintenance execution target is included in the dominant factors of the normal and abnormal risk in the supervision quality of the subsequent maintenance operation of this type of risk analysis terminal. Therefore, operation target compensation processing is carried out for a certain type of risk analysis terminal. Based on the correlation between the abnormal situation of supervision images between maintenance execution targets, the maintenance execution targets corresponding to the certain type of risk analysis terminal are divided. In combination with the actual situation of each compensation target set, the frame extraction execution of the supervision process corresponding to each maintenance execution target in the corresponding compensation target set is set in a targeted manner.

[0069] For a single type of risk analysis terminal, the compensation target set is a set of several maintenance execution targets existing for that type of risk analysis terminal in the next assessment execution cycle of the analysis execution cycle. The interference-related compensation parameter of any compensation target set is greater than a preset interference-related compensation parameter. For a single compensation target set, the interference-related compensation parameter is the average value of the relevant compensation parameters of each maintenance execution target within that compensation target set. For any maintenance execution target within that compensation target set, the relevant compensation parameter of that maintenance execution target... , To determine the number of video frame images for the maintenance task, The number of compensation video frames for the maintenance execution target relative to the compensation target set is defined as follows: For a single video frame of the maintenance execution target, if any other maintenance execution target within the compensation target set exists in the video frame and the corresponding operation occlusion degree is less than or equal to the stage target interference index, then the video frame is recorded as a compensation video frame for the maintenance execution target relative to the compensation target set. For a single compensation target set, the increase in the frame extraction execution parameters of the video monitoring process for each maintenance execution target within the compensation target set compared to the user's initial frame extraction execution parameters is positively correlated with the maintenance operation interference index, where the maintenance operation interference index is the average value of the execution interference parameters of each maintenance execution target within the compensation target set.

[0070] The value of the preset interference-related compensation parameter can be understood as follows: the higher the user's requirements for the quality of supervision of the maintenance execution process of the maintenance personnel, the larger the value of the preset interference-related compensation parameter. The interference-related compensation parameter represents the ability of cross-compensation verification between the video frame information of the maintenance execution targets within the set of maintenance execution targets. A preset interference-related compensation parameter value of 0.3 is provided.

[0071] Specifically, the optimization execution module performs work frame extraction compensation processing on the second-class risk analysis terminal or the first-class risk analysis terminal that has completed the work target compensation processing, and determines the supervision compensation terminal for each maintenance execution target of the work compensation terminal based on the work-related index.

[0072] The job compensation terminal is a type II risk analysis terminal and a type I risk analysis terminal that completes job target compensation processing.

[0073] Specifically, the optimization execution module determines the relevant monitoring terminal for each job compensation terminal based on the execution timing correlation index and the execution space correlation index;

[0074] The execution time correlation index of any of the relevant monitoring terminals to the corresponding job compensation terminals is greater than the preset execution time correlation index, and the execution space correlation index is greater than the preset execution space correlation index.

[0075] For a single Class II risk analysis terminal, since the stage target interference index and stage target cross index corresponding to the Class II risk analysis terminal are both small, it indicates that the image acquisition effect of the image supervision process of the maintenance execution target in the subsequent operation of the Class II risk analysis terminal is of low quality risk. Therefore, there is no need to perform operation target compensation processing for this type of risk analysis terminal. Operation frame extraction compensation processing is performed on the Class II risk analysis terminal and the Class I risk analysis terminal that has completed operation target compensation processing to ensure the supervision quality of the operation behavior of the maintenance personnel corresponding to the risk analysis terminal.

[0076] Two types of risk analysis terminals and one type of risk analysis terminal that completes the work target compensation processing are denoted as work compensation terminals. Work frame extraction compensation processing is performed on each work compensation terminal to determine the supervisory compensation terminal for each maintenance execution target of each work compensation terminal. This supervisory compensation terminal is used to assist in monitoring the corresponding maintenance personnel during maintenance work for each maintenance execution target, ensuring the quality of supervision of the maintenance work process and work safety. For a single work compensation terminal, the relevant supervisory terminal is determined based on the execution time-series correlation index and the execution space correlation index. For a single work monitoring terminal, if the execution time-series correlation index of the work monitoring terminal for the work compensation terminal is greater than a preset execution time-series correlation index and the execution space correlation index is greater than a preset execution space correlation index, the execution time-series correlation index... , The number of maintenance execution targets existing in the next evaluation execution cycle for the analysis execution cycle is determined by the work compensation terminal and the work monitoring terminal. The execution space correlation index represents the number of time-series related targets in the next evaluation execution cycle of the analysis execution cycle for both the job compensation terminal and the job monitoring terminal. , The number of maintenance execution targets existing in the next evaluation execution cycle for the analysis execution cycle is determined by the work compensation terminal and the work monitoring terminal. To determine the number of workspace-related targets in the next evaluation execution cycle of the analysis execution cycle for both the work compensation terminal and the work monitoring terminal, one maintenance execution target is obtained for each terminal. If the work-time correlation index between the two maintenance execution targets is greater than a preset work-time correlation index, then the two maintenance execution targets are recorded as work-time-related targets. If the work-space correlation index between the two maintenance execution targets is greater than a preset work-space correlation index, then the two maintenance execution targets are recorded as work-space-related targets. The work-time correlation index... , To assess the duration corresponding to the execution cycle, The workspace correlation index is the interval between the estimated execution times of the two maintenance execution objectives mentioned above. , This is the shortest distance between the center points of gravity of the two maintenance targets mentioned above. To preset the evaluation distance, for any maintenance execution target of the operation monitoring terminal, the relevant regulatory terminal corresponding to the maximum value of the operation-related index of the maintenance execution target is used as the regulatory compensation terminal for the maintenance execution target. This is used to supplement the video monitoring of the maintenance operation of the maintenance execution target. For a single relevant regulatory terminal, the operation-related index of the maintenance execution target is the maximum value of the sum of the operation time-series correlation index and the operation space correlation index of each maintenance execution target of the relevant regulatory terminal for the maintenance execution target.

[0077] The values ​​of the preset execution timing correlation index and the preset execution space correlation index are readily understood by those skilled in the art. The higher the user's requirements for the quality of supervision of the maintenance execution process by maintenance personnel, the larger the values ​​of the preset execution timing correlation index and the preset execution space correlation index. These indices characterize the temporal and spatial correlation between the maintenance execution targets of the operation monitoring terminal. One example of preset execution timing correlation index and preset execution space correlation index values ​​is provided: the preset execution timing correlation index is 0.6, and the preset execution space correlation index is 0.6. Similarly, another example of preset operation timing correlation index and preset operation timing correlation index values ​​is provided: the higher the user's requirements for the quality of supervision of the maintenance execution process by maintenance personnel, the larger the values ​​of the preset operation timing correlation index and the preset operation space correlation index. Another example of preset operation timing correlation index values ​​is provided: 10, and a preset operation space correlation index value is provided: 1.

[0078] Specifically, the coordinating execution module responds to coordinating execution conditions, performs job coordinating processing for each risk analysis terminal, and determines the coordinating job terminal for each risk analysis terminal based on the execution target overlap index and the execution target cross index.

[0079] The coordinating execution conditions are that the risk target percentage index determined by the analysis execution cycle is less than or equal to the preset risk target percentage index, and the risk-related execution index is less than or equal to the preset risk-related execution index.

[0080] If the risk target proportion index determined by the analysis execution cycle is less than or equal to the preset risk target proportion index and the risk-related execution index is less than or equal to the preset risk-related execution index, it indicates that there are few risk analysis terminals for the current situation and the correlation between the maintenance execution targets corresponding to the existing risk analysis terminals is small. This indicates that the maintenance operation processes of the existing risk analysis terminals are relatively independent. By performing operation coordination processing on each risk analysis terminal, the coordination operation terminal of each risk analysis terminal is determined. Each risk analysis terminal and its determined coordination operation terminal jointly carry out maintenance on the maintenance execution target.

[0081] The process of assigning assistant operation terminals to each risk analysis terminal includes: traversing each risk analysis terminal and selecting the regular operation terminal with the highest assistant selection coefficient as the preferred assistant operation terminal. If a regular operation terminal is selected as the preferred assistant operation terminal by only one risk analysis terminal, then that regular operation terminal is recorded as the assistant operation terminal of that risk analysis terminal. If a regular operation terminal is selected as the preferred terminal by multiple risk analysis terminals simultaneously, then that regular operation terminal is recorded as the assistant operation terminal of the risk analysis terminal corresponding to the highest determined operation risk coefficient. Other risk analysis terminals that fail to obtain a preferred assistant operation terminal are then sequentially selected from the regular operation terminals with the second highest assistant selection coefficients, and the above judgment process is repeated until all risk analysis terminals have been assigned or there are no more regular operation terminals to assign.

[0082] For a single risk analysis terminal, the co-optimization coefficient for each routine operation terminal relative to that risk analysis terminal is determined based on the execution target overlap index and the execution target cross-index. The co-optimization coefficient characterizes the degree of correlation between the two during maintenance operations. For any routine operation terminal, the co-optimization coefficient for that risk analysis terminal is the sum of the products of the execution target overlap index and the execution target cross-index, respectively, and their corresponding optimization weight coefficients. The execution target overlap index... , This refers to the number of maintenance operation targets that coexist with the routine operation terminal and the risk analysis terminal. The number of different maintenance operation targets existing between the routine operation terminal and the risk analysis terminal, the execution target cross index. , This refers to the number of overlapping operational targets between the routine operation terminal and the risk analysis terminal. Given the number of different maintenance operation targets existing between the conventional operation terminal and the risk analysis terminal, the preferred weight coefficients corresponding to the execution target overlap index and the execution target cross index can be set according to the actual working conditions. A preferred weight coefficient for the execution target overlap index and the execution target cross index is provided, with the preferred weight coefficient for both being 0.5.

[0083] Specifically, the frame-skipping execution parameters for each co-executor combination are determined based on the cross-index of the execution target;

[0084] Any of the aforementioned assistant execution combinations is a combination of any risk analysis terminal and its corresponding assistant operation terminal;

[0085] The frame extraction execution parameters of each coordinator execution group are positively correlated with the execution target cross-index between the risk analysis terminal and the coordinator operation terminal within the corresponding coordinator execution group.

[0086] Specifically, for a single coordinating execution group, in the next assessment execution cycle of the analysis execution cycle, the maintenance execution targets existing in the assessment execution cycle for the risk analysis terminal and the coordinating operation terminal within the coordinating execution group are recorded as the maintenance execution targets of the coordinating execution group. The maintenance personnel corresponding to the risk analysis terminal and the coordinating operation terminal are then jointly executing the maintenance tasks corresponding to the maintenance execution targets existing in the two operation monitoring terminals. The frame extraction execution parameters during the maintenance operation process of the two operation monitoring terminals within the coordinating execution group are determined based on the cross-index of the execution targets, ensuring that the acquired video frame information can meet the requirements for the quality supervision of the maintenance operation process. The increase in the newly determined frame extraction execution parameters compared to the user's initial frame extraction execution parameters is positively correlated with the cross-index of the execution targets.

[0087] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A smart vehicle inspection auxiliary system based on real-time monitoring video analysis, characterized in that, include: The operation assessment module is used to conduct operation risk assessment for the vehicle inspection operation process, in order to determine the risk analysis terminal. Based on the risk target proportion index and the risk-related execution index, it determines whether to adjust the operation assistance process to operation risk analysis. The risk analysis terminal is determined according to the operation risk coefficient of each operation monitoring terminal. The risk analysis module, which is connected to the operation evaluation module, is used to perform operation risk analysis for each risk analysis terminal and determine the category of the corresponding risk analysis terminal based on the stage target interference index and stage target cross index of each risk analysis terminal. An optimized execution module, which is connected to the risk analysis module, is used to determine whether the process of performing work frame extraction compensation processing for the corresponding risk analysis terminal includes work target compensation processing based on the category of each risk analysis terminal, to determine the frame extraction execution parameters based on the maintenance work interference index of the compensation target set, and to determine the regulatory compensation terminal for each maintenance execution target of the risk analysis terminal based on the work-related index. The assistant execution module, which is connected to the job evaluation module, is used to perform job assistant processing for each risk analysis terminal to determine the assistant job terminal and frame extraction execution parameters for each risk analysis terminal. The risk target percentage index , The number of risk analysis terminals determined by the analysis execution cycle, The target monitoring terminal is the number of operational monitoring terminals present during the target supervision process. The risk-related execution index is the average of the execution target relevance indices of each risk analysis terminal. For a single risk analysis terminal, the execution target relevance index is... , This refers to the number of relevant execution objectives that exist within the next evaluation execution cycle of the risk analysis terminal during the analysis execution cycle. The number of maintenance execution targets existing in the next evaluation execution cycle of the analysis execution cycle of the risk analysis terminal, wherein the relevant execution targets are maintenance execution targets that exist together with other risk analysis terminals; For a single operation monitoring terminal, the operation risk coefficient is the sum of the products of the abnormal execution index and the stage target interference index of the operation monitoring terminal, respectively, and their corresponding evaluation weight coefficients. The stage target interference index is the average value of the execution interference parameters of each maintenance execution target existing in the next evaluation execution cycle of the current evaluation execution cycle for operation risk coefficient analysis. For a single maintenance execution target, the execution interference parameter is the average value of the operation occlusion degree of each video frame image acquired for that maintenance execution target. The abnormal execution index... , This represents the average vibration amplitude of the monitoring terminal during each instance of abnormal vibration within the evaluation execution cycle. The preset vibration amplitude; For a single risk analysis terminal, the stage target cross index is the average value of the workspace cross parameters of the maintenance execution targets existing in the next evaluation execution cycle of the analysis execution cycle of the risk analysis terminal. For a single compensation target set, the maintenance operation interference index is the average value of the execution interference parameters of each maintenance execution target in the compensation target set. For a single maintenance execution target, the workspace cross parameter is the number of maintenance execution targets existing in the cross evaluation space of the maintenance execution target. For any two maintenance execution objectives, the work sequence correlation index , To assess the duration corresponding to the execution cycle, The workspace correlation index is the interval between the estimated execution times of the two maintenance execution objectives mentioned above. , This is the shortest distance between the center points of gravity of the two maintenance targets mentioned above. To preset the evaluation distance, for any maintenance execution target of the operation monitoring terminal, for a single related regulatory terminal, the operation relevance index for the maintenance execution target is the maximum value of the sum of the operation time sequence relevance index and the operation space relevance index of each maintenance execution target of the related regulatory terminal for the maintenance execution target.

2. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 1, characterized in that, The job evaluation module periodically detects the job risk coefficient of each job monitoring terminal; The risk analysis terminal is a job monitoring terminal with a job risk coefficient greater than a preset job risk coefficient.

3. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 2, characterized in that, The risk analysis module responds to the analysis execution conditions, performs operational risk analysis on each risk analysis terminal, and determines the operational category of each risk analysis terminal; The analysis execution condition is that the risk target percentage index determined in the analysis execution cycle is greater than the preset risk target percentage index or the risk-related execution index is greater than the preset risk-related execution index.

4. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 3, characterized in that, The risk analysis terminals are categorized into Class I risk analysis terminals and Class II risk analysis terminals. The first type of risk analysis terminal is a risk analysis terminal in which the stage target interference index is greater than the preset stage target interference index or the stage target cross index is greater than the preset stage cross index. The second type of risk analysis terminal is a risk analysis terminal where the stage target interference index is less than or equal to the preset stage target interference index and the stage target cross index is less than or equal to the preset stage cross index.

5. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 4, characterized in that, The optimization execution module performs work target compensation processing for a type of risk analysis terminal, and determines the frame extraction execution parameters of the corresponding compensation target set based on the maintenance operation interference index of each compensation target set of the type of risk analysis terminal. The frame extraction execution parameters are positively correlated with the maintenance operation interference index of the compensation target set.

6. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 5, characterized in that, The maintenance operation interference index of any set of compensation targets is determined based on the execution interference parameters of each maintenance execution target within the set of compensation targets; The set of compensation targets is determined based on interference-related compensation parameters; The interference-related compensation parameter is the average value of the relevant compensation parameters of each maintenance execution target within the compensation target set. For any maintenance execution target within the compensation target set, the relevant compensation parameter of that maintenance execution target is... To determine the number of video frame images for the maintenance task, The number of compensation video frame images that exist for the compensation target set for the maintenance execution target.

7. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 6, characterized in that, The optimization execution module performs work frame extraction compensation processing on the second-class risk analysis terminal or the first-class risk analysis terminal that has completed the work target compensation processing, and determines the supervision compensation terminal for each maintenance execution target of the work compensation terminal based on the work-related index. The job compensation terminal is a type II risk analysis terminal and a type I risk analysis terminal that completes job target compensation processing.

8. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 7, characterized in that, The optimized execution module determines the relevant monitoring terminal for each job compensation terminal based on the execution timing correlation index and the execution space correlation index. The execution time correlation index of any of the relevant monitoring terminals to the corresponding job compensation terminals is greater than the preset execution time correlation index and the execution space correlation index is greater than the preset execution space correlation index; For any maintenance execution target of a single operation monitoring terminal, the relevant regulatory terminal corresponding to the maximum value of the operation-related index for that maintenance execution target will be used as the regulatory compensation terminal for that maintenance execution target. For a single job compensation terminal and a single job monitoring terminal, the execution timing correlation index The number of maintenance execution targets existing in the next evaluation execution cycle for the analysis execution cycle is determined by the work compensation terminal and the work monitoring terminal. The execution space correlation index represents the number of time-series related targets in the next evaluation execution cycle of the analysis execution cycle for both the job compensation terminal and the job monitoring terminal. The number of maintenance execution targets existing in the next evaluation execution cycle for the analysis execution cycle is determined by the work compensation terminal and the work monitoring terminal. This refers to the number of workspace-related targets present in the next evaluation execution cycle of the analysis execution cycle for both the work compensation terminal and the work monitoring terminal.

9. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 2, characterized in that, The coordinating execution module responds to coordinating execution conditions, performs job coordinating processing for each risk analysis terminal, and determines the coordinating job terminal for each risk analysis terminal based on the execution target overlap index and the execution target cross index. The coordinating execution condition is that the risk target proportion index determined by the analysis execution cycle is less than or equal to the preset risk target proportion index and the risk-related execution index is less than or equal to the preset risk-related execution index. For any conventional operating terminal, the execution target overlap index This refers to the number of maintenance operation targets that coexist with the routine operation terminal and the risk analysis terminal. The number of different maintenance operation targets existing between the routine operation terminal and the risk analysis terminal, the execution target cross index. , This refers to the number of overlapping operational targets between the routine operation terminal and the risk analysis terminal. This refers to the number of different maintenance operation targets that exist between the routine operation terminal and the risk analysis terminal.

10. The intelligent vehicle inspection auxiliary system based on real-time monitoring video analysis according to claim 9, characterized in that, The frame-skipping execution parameters for each co-executor combination are determined based on the cross-index of the execution target. Any of the aforementioned assistant execution combinations is a combination of any risk analysis terminal and its corresponding assistant operation terminal; The frame extraction execution parameters of each coordinator execution group are positively correlated with the execution target cross-index between the risk analysis terminal and the coordinator operation terminal within the corresponding coordinator execution group.

Images