A motor vehicle inspection supervision system based on video recognition

Abstract

The application provides a motor vehicle inspection supervision system based on video recognition, which can reduce the possibility of data falsification, ensure the accuracy of supervision data, and improve the supervision effect of the supervision system. In the application, based on the original inspection item process information of motor vehicle inspection supervision, the content of the item video recognition is combined with the motor vehicle registration information, bypass inspection information and inspection item information for comparative analysis according to the inspection specification and requirements. Meanwhile, the motor vehicle safety technology inspection process is decomposed according to the inspection items, the starting and ending time of each inspection item is monitored through the inspection business information management system, the time point of supervision implementation is set, after the end of the inspection item, the abnormality recognition and analysis are immediately carried out through the motor vehicle inspection supervision module, the illegal evidence is fixed in the first time, and the problem of missing evidence chain in the after-investigation evidence collection is avoided.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, specifically to a vehicle inspection and supervision system based on video recognition. Background Technology

[0002] Motor vehicle safety technical inspection is an effective means of testing the operational safety performance of motor vehicles and a crucial aspect of motor vehicle management in my country. To standardize processes, a unified national inspection business information management system and a motor vehicle inspection supervision system are implemented. Through system networking between traffic management departments and various inspection agencies, the motor vehicle safety technical inspection supervision system oversees all inspection business information systems. During supervision, video surveillance equipment is installed at inspection stations. An audio-visual acquisition and management system controls the video surveillance equipment to upload video of the entire motor vehicle inspection process. Supervisory personnel monitor the video of the motor vehicle inspection process. The inspection business information management system uploads detailed inspection results and key station inspection process images to the motor vehicle inspection supervision system in real time for review.

[0003] To reduce the workload of manual review during the regulatory process, some cities have adopted intelligent photo review systems for vehicle inspections, using intelligent image recognition and filtering. However, current audio and video surveillance is mostly used in scenarios where static process photos are captured during the inspection and uploaded to the vehicle inspection supervision system for review, and then randomly checked afterward, or reviewed for accountability after anomalies are discovered. While this has deterred violations during vehicle inspections to some extent, the existing systems have not achieved the expected regulatory results in practice due to several issues: the proportion of real-time spot checks is too low, and post-inspections cannot guarantee real-time performance and impartiality; collusion between internal and external parties still exists because the photo review method is technically simplistic, and the cost of falsifying static image data is low. Summary of the Invention

[0004] To address the issue that existing vehicle inspection and supervision systems based on static images fail to achieve the expected regulatory results in practical applications, this invention provides a vehicle inspection and supervision system based on video recognition. This system can reduce the possibility of data falsification, ensure the accuracy of regulatory data, and improve the regulatory effectiveness of the system.

[0005] The technical solution of the present invention is as follows: a motor vehicle inspection and supervision system based on video recognition, comprising: video monitoring equipment and audio and video acquisition and management system, characterized in that it further comprises: a motor vehicle inspection and supervision module, a video recognition module and a dedicated data acquisition device preset at the inspection site;

[0006] The audio and video acquisition and management system controls the video monitoring equipment to realize video monitoring of the entire process of motor vehicle inspection and key inspection stations. At the same time, it segments and retrieves video files based on the inspection vehicle, inspection items, inspection start time and end time of the items written in the inspection business information management system, and supports video and image capture during the inspection process.

[0007] The audio and video acquisition and management system includes: a video storage unit, a video networking unit, a project video file marking unit, and a video stream playback and output unit;

[0008] The video storage unit is used to store and mark the process videos of the inspection items at the inspection station;

[0009] The video networking unit is used to realize interconnection and interoperability between the video private network and the business private network and to enable real-time video playback;

[0010] The project video file marking unit marks the video playback address and format of the inspection project according to the requirements of the inspection business information management system. At the beginning of each inspection project, the inspection business information management system writes the start time of the video file through the project video file marking unit, and at the end of each inspection project, it writes the end time of the video file through the project video file marking unit.

[0011] The video stream playback and output unit is used to output a video stream;

[0012] The video recognition module identifies the monitoring video data based on the video address and video type to obtain the video recognition result;

[0013] During the vehicle inspection process, the dedicated data collector pushes the collected data as bypass inspection information to the vehicle inspection supervision module according to the preset bypass data inspection items. The bypass data inspection items are the inspection items of the vehicle to be confirmed, other than video and audio that cannot be monitored by the video monitoring equipment. The bypass inspection information is transmitted to the vehicle inspection supervision module via a dedicated line.

[0014] The motor vehicle inspection supervision module reads the inspection process information uploaded by the inspection business information management system. Based on the inspection process information, it compares and analyzes the inspection process information with the video recognition result of the monitoring video data corresponding to the inspection item and the bypass data inspection information according to the inspection item confirmation and identification content. This enables real-time monitoring of motor vehicle inspection process data and identification and alarm processing of abnormal data.

[0015] Before the inspection of the motor vehicle to be confirmed begins, the inspection business information management system collects the motor vehicle information of the motor vehicle to be confirmed and compares it with the motor vehicle registration information of the motor vehicle to be confirmed through the motor vehicle inspection supervision module. The comparison result is stored as the unique comparison result information of the motor vehicle to be confirmed.

[0016] The motor vehicle inspection supervision module includes: an inspection item start receiving unit, an inspection process image receiving unit, an inspection detailed result receiving unit, an inspection item end receiving unit, a video result analysis and processing unit, a video result analysis and processing unit, and a data anomaly processing unit;

[0017] The inspection item start receiving unit is used to receive inspection item start information and legality verification; when starting each inspection item, the inspection business information management system writes the inspection item start information through the inspection item start receiving unit.

[0018] The inspection process image receiving unit is used to receive photos captured by the video monitoring equipment during the project inspection process; during the implementation of the inspection project, the inspection business information management system will write randomly captured motor vehicle inspection process photos into inspection photo information through the inspection process image receiving unit.

[0019] The detailed inspection result receiving unit receives all inspection process information uploaded by the inspection business information management system after each inspection item is completed. The inspection process information includes all inspection item information and the corresponding identification results of the inspection item. In the inspection process information, all data is stored and marked separately for each inspection item.

[0020] The inspection project completion receiving unit is used to receive inspection project completion information and legality verification; after each inspection project is completed, the inspection business information management system writes the project completion information through the inspection project completion receiving unit.

[0021] The system scheduling and control unit is used to send a video recognition calculation request to the video recognition module after the project inspection is completed, based on the inspection vehicle, inspection items, and inspection start and end times.

[0022] The video result analysis and processing unit receives the video recognition results uploaded by the video recognition module according to preset inspection specifications and requirements. After the inspection item is completed, it compares the video recognition results and the bypass inspection information with the recognition results of the corresponding inspection items in the respective inspection process information. If the data error in the comparison results exceeds a set threshold or the comparison results are inconsistent, an early warning is issued, and the corresponding inspection item and comparison results are transmitted to the data anomaly processing unit.

[0023] The data anomaly processing unit is used to screen items that show anomalies during the inspection process and initiate an anomaly investigation and handling process.

[0024] Its further features are:

[0025] The video surveillance equipment and other modules are set up in a network environment that is isolated from each other. There is no unified clock server between different network environments. The clock between modules is calibrated and calculated based on a clock compensation algorithm.

[0026] The clock compensation algorithm includes the following steps:

[0027] a1: The audio and video acquisition and management system uploads the data to be calibrated to the inspection business information management system.

[0028] The data to be calibrated includes: video data captured by video surveillance equipment and the video data acquisition and upload time;

[0029] a2: The inspection business information management system sends the data to be calibrated to the motor vehicle inspection supervision module;

[0030] a3: The motor vehicle inspection and supervision module transmits the data to be calibrated to the video recognition module;

[0031] a4: The video recognition module identifies the shooting time of the video data based on the image recognition algorithm, compares the shooting time with the video data collection and upload time to calculate the time difference, and returns the time difference to the motor vehicle inspection and supervision module;

[0032] a5: The actual shooting time of each video data is calculated by the time difference and the video data collection and upload time of the motor vehicle inspection and supervision module;

[0033] Step a4 includes the following steps in detail:

[0034] a41: Image noise reduction processing;

[0035] Image frames are obtained based on the video data and denoted as: image data to be identified; noise reduction processing is performed on each frame of the image data to be identified.

[0036] a42: The image data to be identified is scaled according to a preset size and then normalized;

[0037] a43: Constructing a text detection model based on the DBnet model;

[0038] The text detection model takes the image data to be recognized as input and outputs the coordinate positions of the text in the image data to be processed.

[0039] a44: The image data to be identified is fed into the text detection model to obtain the corresponding text coordinate position in the image data to be identified;

[0040] a45: Based on the text coordinate position, each of the images to be identified is cropped to obtain a text region image;

[0041] The text region image is scaled and normalized according to a specified size to obtain the text image to be recognized;

[0042] a46: Constructing a text recognition model based on the CRNN model;

[0043] The data for the text recognition model is the text image to be recognized, and the output is text characters.

[0044] a47: Input the text image to be recognized into the text recognition model to obtain the corresponding text characters;

[0045] The text characters are converted into time data according to a preset time format, thus obtaining the shooting time;

[0046] a48: Subtract the time between the shooting from the time of the video data collection and upload to obtain the time difference;

[0047] The inspection business information management system and the motor vehicle inspection supervision module communicate via an external software interface authorization request method; the inspection process information is stored using key encryption to prevent third parties from tampering with the process information;

[0048] The motor vehicle inspection and supervision module and the video recognition module exchange data using an encrypted signature method. Asymmetric key pairs are generated for different algorithm providers. When the motor vehicle inspection and supervision module initiates a video detection request, it signs the request parameters with its private key. The video recognition algorithm calculation module verifies the signature information using its public key. When the video recognition module writes the recognition result back to the motor vehicle inspection and supervision module, it uses an external software interface to apply for authorization.

[0049] The video recognition module includes: a video reading unit, a video recognition calculation unit, and a video result encapsulation and processing unit;

[0050] After receiving the video recognition calculation request initiated by the system scheduling and control unit, the video reading unit reads the video stream stored in the audio and video acquisition management system through the video address and sends it to the video recognition calculation unit.

[0051] The video recognition calculation unit performs frame extraction processing on the video stream, performs online video recognition according to the video type, performs computer video calculations on the video according to the recognition project requirements, and sends the generated calculation results to the video result encapsulation processing unit.

[0052] The video result encapsulation and processing unit encapsulates and packages the video recognition results in JSON format and uploads them to the video result analysis and processing unit in the motor vehicle inspection and supervision module for further processing.

[0053] The video storage unit supports two management methods for video surveillance equipment:

[0054] Management Method 1: Each testing line is equipped with a workstation camera. The workstation program sends start and end signals to the corresponding workstation of the testing line, and the workstation notifies the workstation camera to take video and photos. After all videos are taken, they are stored in the WEB video server according to the preset retention period.

[0055] Management Method 2: All video surveillance cameras are connected to the hard disk recorder, configured with corresponding ports, and the start and end times of each workstation are recorded. Based on the port and time, video data stored in the hard disk recorder is extracted and stored in the WEB video server according to the preset retention period. For video storage requirements of the global perspective, non-workstation video data of the corresponding perspective is obtained and stored in the hard disk recorder according to the preset retention period.

[0056] In the video recognition module, when the type of the motor vehicle to be confirmed is a flatbed truck and a flatbed trailer, the video recognition result includes: the height of the cargo box sideboards;

[0057] The steps for identifying the height of the cargo box side panels include:

[0058] b1: Construct a focus estimation model based on CenterNet;

[0059] The input to the focus estimation model is the image data to be identified, and the output is the positions of the four corner points of the license plate of the vehicle to be identified and the positions of the four focal points of the rear panel of the vehicle.

[0060] b2: Calculate the width and height of the license plate based on the four corner points, and record it as: license plate recognition dimensions;

[0061] The width and height of the cargo box side panel are calculated based on the four focal points of the rear side panel of the vehicle and recorded as: cargo box side panel identification dimensions;

[0062] b3: Obtain the standard dimensions of the license plate, denoted as: standard license plate dimensions;

[0063] b4: Compare the standard license plate size with the license plate recognition size;

[0064] Let w be the ratio of the width in the standard license plate dimensions to the width in the license plate recognition dimensions, and h be the ratio of the height.

[0065] b5: Based on w and h, the actual width and height of the rear side panel of the vehicle are calculated by using the width and height in the dimensions of the cargo box side panel.

[0066] This invention provides a vehicle inspection and supervision system based on video recognition. Building upon the original inspection process information, it significantly improves the ability to identify abnormal situations during the inspection process by combining video recognition of the inspection items with vehicle registration information, bypass inspection information, and inspection item information, based on inspection standards and requirements. This approach breaks down the vehicle safety technical inspection process into inspection items, monitors the start and end times of each item through an inspection business information management system, sets the time points for supervision implementation, and immediately identifies and analyzes anomalies after each inspection item is completed. This ensures that evidence of violations is secured as soon as possible, avoiding the problem of missing evidence chains in subsequent investigations. Based on this technical solution, the possibility of data falsification can be reduced, ensuring the accuracy of regulatory data and improving the effectiveness of the supervision system. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the system module structure of the video recognition-based motor vehicle inspection and supervision system in this application;

[0068] Figure 2 A schematic diagram showing the relationships between software modules in the motor vehicle inspection and supervision system;

[0069] Figures 3-6 Examples of inspection items for motor vehicle inspection;

[0070] Figure 7 Here is an example of a video frame from video data collected by a video surveillance device;

[0071] Figure 8 This is a flowchart of text region recognition based on the DBnet model;

[0072] Figure 9 This is an example of text recognition based on a CRNN model;

[0073] Figure 10 This is an example of identifying the four corner points of a vehicle and its license plate based on the CenterNet model. Detailed Implementation

[0074] like Figure 1 As shown, the present invention includes a motor vehicle inspection and supervision system based on video recognition, which includes: video monitoring equipment, audio and video acquisition and management system, motor vehicle inspection and supervision module, video recognition module, and a dedicated data acquisition device preset at the inspection site.

[0075] Video surveillance equipment is installed at workstations throughout the entire vehicle inspection process.

[0076] During the vehicle inspection process, the dedicated data collector pushes the collected data as bypass inspection information to the vehicle inspection supervision module according to the preset bypass data inspection items. The bypass data inspection items are the inspection items of the vehicle to be confirmed, other than video and audio that cannot be monitored by video monitoring equipment, such as braking inspection items and light intensity inspection items. The bypass inspection information is transmitted to the vehicle inspection supervision module through a dedicated line.

[0077] In practice, the dedicated data acquisition device is uniformly configured to ensure that the original data during the data verification process cannot be tampered with. It collects bypass verification information such as braking force data and headlight intensity data. This data is then compared with the verification data uploaded by the verification agency to ensure the authenticity of the verification data.

[0078] The audio and video acquisition and management system controls video surveillance equipment to achieve video monitoring of the entire vehicle inspection process and key inspection stations. It also supports the storage, tagging, and retrieval of video information during the inspection process, and allows for video capture during inspections. It provides an interface for storing or tagging video files by inspection item, and enables the capture and retrieval of video photos, as well as the acquisition of video streams based on the inspected vehicle, inspection item, and the start and end times of the inspection.

[0079] like Figure 2 As shown, the audio and video acquisition and management system includes: a video storage unit, a video networking unit, a project video file marking unit, and a video stream playback and output unit.

[0080] The video storage unit is used to store and label the inspection process videos at the inspection station as needed.

[0081] For video surveillance equipment, the video storage unit supports two management methods:

[0082] Management Method 1: Each testing line is equipped with a workstation camera. The workstation program sends start and end signals to the corresponding workstation of the testing line, and the workstation notifies the workstation camera to take video and photos. After all videos are taken, they are stored in the WEB video server according to the preset retention period.

[0083] Management Method 2: All video surveillance cameras are connected to a hard disk recorder, configured with corresponding ports, and the start and end times of each workstation are recorded. Based on the port and time, video data stored in the hard disk recorder is extracted and saved in the WEB video server according to the preset retention period. For video storage requirements of the global perspective, non-workstation video data of the corresponding perspective is obtained and stored in the hard disk recorder according to the preset retention period.

[0084] The video networking unit enables interconnection and real-time video playback between the video private network and the business private network through the core switch;

[0085] The project video file marking unit marks video files according to the inspection items and start and end times, as required by the inspection business information management system, to facilitate video retrieval and reading. The inspection business information management system writes the start time of the video file at the beginning of each inspection item and writes the end time of the video file at the end of each inspection item through the project video file marking unit.

[0086] The video stream playback and output unit is used to output video streams. The video data playback and output methods supported by the video stream playback and output unit include:

[0087] The web video server provides a remote URL access address.

[0088] It acquires a specified video stream in RTSP format and performs intelligent recognition and output.

[0089] The audio and video acquisition and management system segments and retrieves video files based on the inspection vehicle, inspection item, and the start and end times of the inspection item as markers written in the inspection business information management system. This provides a data foundation for resolving the inconsistency between video information and inspection process text information caused by clock inconsistencies between different systems.

[0090] The motor vehicle inspection supervision module reads the inspection process information uploaded by the inspection business information management system. Based on the inspection process information, it confirms and identifies the content according to the inspection items. It then compares and analyzes the inspection process information with the video recognition results of the monitoring video data corresponding to the inspection items and the bypass data inspection information to achieve real-time monitoring of motor vehicle inspection process data and identification and alarm processing of abnormal data.

[0091] Before the inspection of a motor vehicle to be confirmed begins, the inspection business information management system collects the motor vehicle information of the motor vehicle to be confirmed and compares it with the motor vehicle registration information of the motor vehicle to be confirmed through the motor vehicle inspection supervision module. The comparison result is stored as the unique comparison result information of the motor vehicle to be confirmed.

[0092] The motor vehicle inspection supervision module includes: an inspection item start receiving unit, an inspection process image receiving unit, an inspection detailed result receiving unit, an inspection item end receiving unit, a video result analysis and processing unit, a video result analysis and processing unit, and a data anomaly handling unit;

[0093] The inspection item start receiving unit is used to receive inspection item start information and legality verification. When starting each inspection item, the inspection business information management system writes the inspection item start information through the inspection item start receiving unit.

[0094] The inspection process image receiving unit is used to receive photos captured by video monitoring equipment during the project inspection process. During the implementation of the inspection project, the inspection business information management system will randomly take photos of the motor vehicle inspection process and write them into the inspection photo information through the inspection process image receiving unit.

[0095] The detailed inspection result receiving unit receives all inspection process information uploaded by the inspection business information management system after each inspection item is completed. The inspection process information includes all inspection item information and the corresponding identification results of the inspection items. In the inspection process information, all data is stored and marked separately for each inspection item.

[0096] The inspection project completion receiving unit is used to receive inspection project completion information and verify legality. After each inspection project is completed, the inspection business information management system writes the project completion information through the inspection project completion receiving unit.

[0097] The system scheduling and control unit is used to send a video recognition calculation request to the video recognition module after the project inspection is completed, based on the inspection vehicle, inspection items, and the start and end times of the inspection.

[0098] The video result analysis and processing unit receives the video recognition results uploaded by the video recognition module according to preset inspection specifications and requirements. After the inspection item is completed, it compares the video recognition results and bypass inspection information with the recognition results of the corresponding inspection items in the separate inspection process information. If the data error in the comparison results exceeds the set threshold or the comparison results do not match, an early warning is issued, and the corresponding inspection item and comparison results are transmitted to the data anomaly processing unit.

[0099] The data anomaly handling unit is used to screen items that have anomalies in the inspection process and initiate anomaly investigation and handling procedures.

[0100] In the motor vehicle inspection supervision module, the steps to achieve real-time monitoring of motor vehicle inspection process data and automatic alarm processing of abnormal data include:

[0101] d1: Receive motor vehicle information uploaded by the inspection business information management system, compare it with the motor vehicle technical parameters and registration information, and confirm its uniqueness.

[0102] d2: Receives real-time data from the inspection business information management system, including inspection items, start time, captured photos, detailed information, and end time. It also receives bypass inspection information from a dedicated data acquisition unit in real-time and merges and saves detailed inspection results.

[0103] d3: After the project inspection is completed, the system scheduling and control unit is triggered. Based on the inspected vehicle, inspection items, and the start and end times of the inspection, a video recognition calculation request is initiated.

[0104] d4: Asynchronously wait to receive video recognition and processing results information uploaded by the audio and video acquisition and management system, and save the video recognition and detection content information.

[0105] d5: The background task reads detailed inspection results, bypass inspection information, and video recognition results. It performs multi-channel data comparison and verification, and issues warnings if data errors exceed the set threshold or if the comparison results are inconsistent.

[0106] d6: After an anomaly warning is issued, video data and inspection result data can be accessed for manual comparison and review. If any issues are found, the issuance of the inspection qualification mark will be cancelled.

[0107] In this application, the inspection business information management system realizes business process management of the entire process of motor vehicle inspection, collects inspection process information, and stores and marks the corresponding data of all inspection items included in the motor vehicle safety technical inspection according to the inspection items.

[0108] In practice, the verification of the business information management system includes:

[0109] The vehicle uniqueness verification unit is used to collect and inspect vehicle information and compare it with the unique information of motor vehicles;

[0110] The inspection project start processing unit is used to record and upload inspection project start instructions;

[0111] The inspection process image capture unit is used to capture and upload photos of the inspection process.

[0112] The project inspection details unit is used for collecting and uploading detailed results of the inspection project;

[0113] The inspection project completion processing unit is used to record and upload inspection project completion instructions.

[0114] The inspection business information management system monitors each vehicle awaiting verification through the following steps:

[0115] c1: Monitor each motor vehicle entering the safety technical inspection process in real time and record it as a motor vehicle to be confirmed;

[0116] c2: Collect the motor vehicle information of the motor vehicle to be confirmed, and automatically compare it with the motor vehicle registration information of the motor vehicle to be confirmed through the motor vehicle inspection and supervision module interface. Store the comparison result of the motor vehicle inspection and supervision module as the unique comparison result information of the motor vehicle to be confirmed.

[0117] If the information is inconsistent, stop the inspection; otherwise, proceed to step c3 for the vehicle to be confirmed.

[0118] c3: When starting each inspection item, write the inspection item start information through the interface provided by the inspection item start receiving unit; simultaneously write the video file start time through the interface provided by the project video file marking unit.

[0119] c4: During the implementation of each inspection item, photos of the motor vehicle inspection process are randomly taken by video monitoring equipment and written into the inspection photo information through the interface provided by the inspection process image information receiving unit.

[0120] c5: After the inspection project is completed, the inspection business information module writes the video file end time through the interface provided by the project video file marking unit; and writes the inspection project information and project end information through the interface provided by the inspection project end receiving unit.

[0121] c6: Repeat steps c1 to c4 until all items corresponding to the motor vehicle to be confirmed have been inspected and the inspection completion information is written through the motor vehicle inspection supervision module.

[0122] The inspection business information management system and the motor vehicle inspection supervision module communicate via an external software interface for authorization; inspection process information is stored using key encryption to prevent third parties from tampering with the process information.

[0123] The vehicle inspection and supervision module and the video recognition module exchange data using encrypted signatures. Asymmetric key pairs are generated for different algorithm providers. When the vehicle inspection and supervision module initiates a video detection request, it signs the request parameters with its private key. The video recognition algorithm calculation module verifies the signature information using its public key. When the video recognition module writes the recognition results back to the vehicle inspection and supervision module, it uses an external software interface to apply for authorization. This ensures the security of the entire supervision process and the fairness of the results.

[0124] In practical applications, because the inspection business information management system and the motor vehicle inspection supervision system operate in different local area networks (LANs) within a dedicated network, the video surveillance equipment, audio and video acquisition management system, and inspection business information management system are located on the same LAN, while other modules of the motor vehicle inspection supervision system operate on a separate LAN. In this dedicated network environment, there is no unified clock server between these different LANs. This leads to inconsistencies between the recording times of the front-end video surveillance equipment and the times of other business processing modules within the motor vehicle inspection supervision system. This time inconsistency between different modules within the same motor vehicle inspection supervision system affects video retrieval efficiency and causes inaccurate matching between marked videos and business data, resulting in low accuracy of video recognition and detection results. Therefore, this application uses a clock compensation algorithm to calibrate the clocks between modules.

[0125] The clock compensation algorithm includes the following steps:

[0126] a1: The audio and video acquisition and management system uploads the data to be calibrated to the inspection business information management system.

[0127] The data to be calibrated includes: video data captured by video surveillance equipment and the video data acquisition and upload time;

[0128] a2: Send the data to be calibrated to the motor vehicle inspection supervision module through the inspection business information management system;

[0129] a3: The motor vehicle inspection and supervision module transmits the data to be calibrated to the video recognition module;

[0130] a4: The video recognition module is based on image recognition algorithms, such as OCR technology, to identify the shooting time of video data, compare the shooting time with the video data collection and upload time, calculate the time difference, and return the time difference to the motor vehicle inspection and supervision module.

[0131] a5: The actual recording time of each video data is calculated by combining the time difference of the motor vehicle inspection and supervision module with the video data collection and upload time.

[0132] For example: The audio and video acquisition and management system is recording an inspection item for the left front scene of the appearance inspection. The inspection business information management system obtains video frame data from the video stream of the audio and video acquisition and management system according to the video start recording time (2022-09-01 12:25:00). Figure 7 The image data to be identified in this embodiment is shown below. The image data to be identified is also uploaded to the inspection business information management system.

[0133] The inspection business information management system uploads the image data to be identified, the video start recording time, and the video end recording time to the inspection supervision system.

[0134] The motor vehicle inspection and supervision module sends the image data to be identified and the video upload time to the video recognition module.

[0135] The video recognition module uses OCR technology and a text detection model built on the DBnet model to identify the time in the image. It also calculates the time difference based on the upload time and returns it to the vehicle inspection and supervision module. In this embodiment, the calculated time difference is 1 minute and 38 seconds.

[0136] The motor vehicle inspection and supervision module calculates the actual start and end times of video recording in the audio and video acquisition and management system based on the time difference and the video start recording time, and then sends the correct video to the video recognition module.

[0137] like Figure 8 As shown, the text image to be recognized is input as an image into the DBnet model. The trained DBnet model has complete image mapping capabilities. Based on the threshold image, the input text image to be recognized is segmented to obtain a segmentation image. In this application, a text detection model is constructed based on the DBnet model to recognize the time in the image data to be recognized. The DBNet algorithm is adopted. Based on the segmentation algorithm, the concept of differentiable binarization is proposed: that is, adaptive binarization is performed on each pixel. The binarization threshold is learned by the network. The binarization step is completely added to the network for training. In this way, the final output image has high robustness to the threshold, thereby ensuring the accuracy of the detection results.

[0138] Step a4 includes the following steps in detail:

[0139] a41: Image noise reduction processing;

[0140] Image frames are obtained from video data and denoted as: image data to be identified; each frame of image data to be identified is subjected to noise reduction processing;

[0141] a42: The image data to be recognized is scaled according to a preset size and then normalized.

[0142] a43: Constructing a text detection model based on the DBnet model;

[0143] The text detection model takes the image data to be recognized as input, which is the high-dimensional array data that has been scaled and normalized in step a42, and outputs the coordinate position of the text in the image data to be processed.

[0144] In practical applications, the training data for the constructed DBNet model comes from two parts: one part is data obtained from video business scenarios, and the other part is data synthesized by writing a text image generation program. The real data consists of approximately 2000 images, and the synthesized data contains 50,000 images. The real images are manually labeled, while the synthesized images have built-in annotation information. All this data is used to train the DBNet text detection model. During model training, the Adam optimization strategy is adopted, with a learning rate set to 0.001. Setting the learning rate too high may prevent convergence, while setting it too low will result in very slow convergence.

[0145] a44: Input the image data to be recognized into the text detection model to obtain the corresponding text coordinate position in the image data to be recognized;

[0146] a45: Based on the text coordinate position, each image data to be recognized is cropped to obtain a text region image; because the time display position in the video image captured by the video surveillance equipment used in this application is fixed, the position of the time in the frame can be determined by directly defining the text coordinate position, which will not be confused with other text regions in the video, and there is no need to recognize and check other text, which greatly improves the calculation efficiency; the text region image is scaled and normalized according to the specified size to obtain the text image to be recognized;

[0147] a46: Constructing a text recognition model based on the CRNN model;

[0148] The text recognition model takes the text image to be recognized as its data and outputs text characters, realizing the recognition process from text in an image to editable text characters;

[0149] a47: Input the text image to be recognized into the text recognition model to obtain the corresponding text characters;

[0150] The text characters are converted into time data according to the preset time format, which gives the shooting time;

[0151] a48: Subtract the time between shooting from the time of video data collection and upload to obtain the time difference.

[0152] Text recognition is a method for predicting sequences, so it uses an RNN network for sequence prediction. After extracting features from the image using a CNN, an RNN is used to predict the sequence, and finally, a CTC translation layer is used to obtain the final result.

[0153] The CRNN algorithm mainly uses a three-layer network structure of CNN+RNN+CTC, which are as follows from bottom to top:

[0154] (1) Convolutional layer, using CNN, to extract feature sequences from the input image;

[0155] (2) Recurrent layer: using RNN to predict the label (true value) distribution of the feature sequence obtained from the convolutional layer;

[0156] (3) Transcription layer: Using CTC, the tag distribution obtained from the circulating layer is converted into the final recognition result through operations such as deduplication and integration.

[0157] like Figure 9 As shown, date recognition involves inputting the text image to be recognized into a convolutional layer, extracting the feature sequence, and then performing prediction processing through a recurrent layer to obtain the corresponding text characters. After processing through a transcription layer, the final recognition result is obtained, which is the text characters output according to a preset format: 09-01-2022. The process for recognizing specific times is the same as the process for recognizing dates.

[0158] The video recognition module identifies the surveillance video data based on the video address and video type, and obtains the video recognition result.

[0159] The video recognition module includes: a video reading unit, a video recognition calculation unit, and a video result encapsulation and processing unit;

[0160] After receiving the video recognition calculation request initiated by the system scheduling and control unit, the video reading unit sends the video stream stored in the audio and video acquisition management system through the video address to the video recognition calculation unit, based on the vehicle being inspected, the inspection items, and the start and end times of the inspection.

[0161] The video recognition computing unit performs frame extraction processing on the video stream, and performs online video recognition according to the video type, following the procedure as follows: Figures 3-6 The recognition project requirements shown are to perform computer video calculations on the video and send the generated calculation results to the video result encapsulation and processing unit;

[0162] The video recognition computing unit identifies the following information: appearance, inspection time, reflective markings, protective devices, lights, license plate number, enlarged license plate number, displacement, staff, driving area range, and location.

[0163] The video result encapsulation and processing unit encapsulates and packages the video recognition results in JSON format and uploads them to the video result analysis and processing unit in the motor vehicle inspection and supervision module for further processing.

[0164] In the video recognition module, when the type of vehicle to be identified is a flatbed truck or a flatbed trailer, the video recognition result includes the height of the cargo box sideboard. Traditional methods of calculating height using a "cargo box sideboard ruler" as a reference point suffer from significant calculation errors due to uncertainties in the video camera's shooting angle and distance. This application's technical solution reduces the error rate by using the vehicle license plate as a reference point for height calculation. Vehicle license plates are standard components, with length and width conforming to national standards. By comparing the license plate from the same video with the height to be calculated, the problem of large height calculation errors can be solved more accurately. Furthermore, it eliminates the need to rely on the standard dimensions of the license plate for calculation and avoids considerations such as the image's shooting angle, resulting in faster calculation speed and higher efficiency.

[0165] The steps for identifying the height of the cargo box side panels include:

[0166] c1: Construct a focus estimation model based on CenterNet;

[0167] The input to the focus estimation model is the image data to be identified, and the output is the positions of the four corner points of the license plate of the vehicle to be identified and the positions of the four foci of the rear panel of the vehicle.

[0168] c2: Calculate the width and height of the license plate based on the four corner points, and record it as: license plate recognition dimensions;

[0169] The width and height of the cargo box side panel are calculated based on the four focal points of the rear side panel of the vehicle and recorded as: cargo box side panel identification dimensions;

[0170] c3: Obtain the standard size of the license plate, denoted as: standard size of license plate;

[0171] c4: Compare standard license plate size with license plate recognition size;

[0172] Let w be the ratio of the width in the standard license plate dimensions to the width in the license plate recognition dimensions, and h be the ratio of the height.

[0173] c5: Based on w and h, the actual width and height of the rear side panel of the vehicle are calculated by identifying the width and height in the dimensions of the cargo box side panel.

[0174] like Figure 10 As shown, the image data to be identified is scaled and normalized before being fed into the CenterNet network. The network then obtains the four corner points of the vehicle and its license plate, as shown in the figure.

[0175] Based on the coordinates of the four corner points of the license plate in the image as [[1959,2235],[2197,2245],[1951,2360],[2195,2369]], and the four corner points of the rear side panel of the truck as [[1452,298],[2951,618],[2755,2230],[1332,2149]], the width of the license plate in the image is calculated to be 38, and the width of the truck body is 499. Since the actual license plate has a standard width and height, with a standard width of 45cm, based on the above data, the actual width of the rear side panel of the truck is calculated to be 590.9cm.

[0176] This invention provides a vehicle inspection and supervision system based on video recognition. It enables intelligent online monitoring of video information from the entire process of vehicle inspection, including exterior inspection, chassis inspection, in-line inspection, and road testing. This provides technical support for addressing prominent issues such as unauthorized reduction of inspection items, lowering of inspection standards, and tampering / falsification of inspection data. Simultaneously, it can further free up limited police resources, fully leveraging the intuitiveness, accuracy, timeliness, and rich information content of audio-visual monitoring systems to conduct targeted spot checks and playback. This technical solution adds video recognition content to the original inspection process information, combined with vehicle registration information, detailed inspection results, and bypass inspection information. Based on inspection specifications and requirements, it compares and analyzes the inspection results and detailed information with the results obtained from video recognition, ensuring that inspection personnel conduct safe inspections of vehicles according to inspection procedures and standard operating actions, avoiding non-inspection or inspections not performed according to operating procedures. Furthermore, the integrated application of license plate recognition, enlarged license plate recognition, and vehicle exterior recognition can, to some extent, prevent substitute inspections. Through the systematic promotion and application of video recognition and real-time anomaly detection and early warning technologies, a large amount of manual photo review can be gradually replaced to some extent, which is conducive to gradually transforming the work mode, improving work efficiency, and standardizing management. In the technical solution of this application, the application of AI video recognition technology can enhance the performance capabilities of supervisory and management departments, promptly detect and intervene in abnormal behaviors such as substitute inspections, lowering inspection standards, and failure to follow prescribed procedures during the inspection process. Artificial intelligence technology based on video surveillance can effectively prevent image forgery. At the same time, the application of AI technology can improve work efficiency, promptly detect potential data forgery problems, and strengthen supervisory and management responsibilities.

Claims

1. A vehicle inspection and supervision system based on video recognition, comprising: The video surveillance equipment and audio-visual acquisition and management system are characterized in that they further include: a motor vehicle inspection and supervision module, a video recognition module, and a dedicated data acquisition device pre-installed at the inspection site; The audio and video acquisition and management system controls the video monitoring equipment to realize video monitoring of the entire process of motor vehicle inspection and key inspection stations. At the same time, it segments and retrieves video files based on the inspection vehicle, inspection items, inspection start time and end time of the items written in the inspection business information management system, and supports video and image capture during the inspection process. The audio and video acquisition and management system includes: a video storage unit, a video networking unit, a project video file marking unit, and a video stream playback and output unit; The video storage unit is used to store and mark the process videos of the inspection items at the inspection station; The video networking unit is used to realize interconnection and interoperability between the video private network and the business private network and to enable real-time video playback; The project video file marking unit marks the video playback address and format of the inspection project according to the requirements of the inspection business information management system. At the beginning of each inspection project, the inspection business information management system writes the start time of the video file through the project video file marking unit, and at the end of each inspection project, it writes the end time of the video file through the project video file marking unit. The video stream playback and output unit is used to output a video stream; The video recognition module identifies the monitoring video data based on the video address and video type to obtain the video recognition result; During the vehicle inspection process, the dedicated data collector pushes the collected data as bypass inspection information to the vehicle inspection supervision module according to the preset bypass data inspection items. The bypass data inspection items are the inspection items of the vehicle to be confirmed, other than the video and audio that cannot be monitored by the video monitoring equipment. The bypass inspection information is transmitted to the vehicle inspection supervision module from a dedicated line. The motor vehicle inspection supervision module reads the inspection process information uploaded by the inspection business information management system. Based on the inspection process information, it confirms and identifies the inspection items and compares and analyzes the inspection process information with the video recognition results of the monitoring video data corresponding to the inspection items and the bypass inspection information. This enables real-time monitoring of motor vehicle inspection process data and identification and alarm processing of abnormal data. Before the inspection of the motor vehicle to be confirmed begins, the inspection business information management system collects the motor vehicle information of the motor vehicle to be confirmed, and compares it with the motor vehicle registration information of the motor vehicle to be confirmed through the motor vehicle inspection supervision module. The comparison result is stored as the unique comparison result information of the motor vehicle to be confirmed. The motor vehicle inspection supervision module includes: an inspection item start receiving unit, an inspection process image receiving unit, an inspection detailed result receiving unit, an inspection item end receiving unit, a system scheduling and control unit, a video result analysis and processing unit, and a data anomaly processing unit; The inspection item start receiving unit is used to receive inspection item start information and legality verification; when starting each inspection item, the inspection business information management system writes the inspection item start information through the inspection item start receiving unit. The inspection process image receiving unit is used to receive photos captured by the video monitoring equipment during the project inspection process; during the implementation of the inspection project, the inspection business information management system will write randomly captured motor vehicle inspection process photos into inspection photo information through the inspection process image receiving unit. The detailed inspection result receiving unit receives all inspection process information uploaded by the inspection business information management system after each inspection item is completed. The inspection process information includes all inspection item information and the corresponding identification results of the inspection item. In the inspection process information, all data is stored and marked separately for each inspection item. The inspection project completion receiving unit is used to receive inspection project completion information and legality verification; after each inspection project is completed, the inspection business information management system writes the project completion information through the inspection project completion receiving unit. The system scheduling and control unit is used to send a video recognition calculation request to the video recognition module after the project inspection is completed, based on the inspection vehicle, inspection items, and inspection start and end times. The video result analysis and processing unit receives the video recognition result uploaded by the video recognition module according to the preset inspection specifications and requirements. After the inspection item is completed, the video recognition result and the bypass inspection information are compared with the recognition results of the corresponding inspection items in the inspection process information. If the data error in the comparison result exceeds the set threshold or the comparison result is inconsistent, an early warning is issued, and the corresponding inspection item and comparison result are transmitted to the data anomaly processing unit. The data anomaly processing unit is used to screen items that show anomalies during the inspection process and initiate an anomaly investigation and handling process.

2. The vehicle inspection and supervision system based on video recognition according to claim 1, characterized in that: The video surveillance equipment and other modules are set up in a network environment that is isolated from each other. There is no unified clock server between different network environments. The clock between modules is calibrated and calculated based on a clock compensation algorithm. The clock compensation algorithm includes the following steps: a1: The audio and video acquisition and management system uploads the data to be calibrated to the inspection business information management system; The data to be calibrated includes: video data captured by video surveillance equipment and the video data acquisition and upload time; a2: The inspection business information management system sends the data to be calibrated to the motor vehicle inspection supervision module; a3: The motor vehicle inspection and supervision module transmits the data to be calibrated to the video recognition module; a4: The video recognition module identifies the shooting time of the video data based on the image recognition algorithm, compares the shooting time with the video data collection and upload time to calculate the time difference, and returns the time difference to the motor vehicle inspection and supervision module; a5: The actual shooting time of each video data is calculated by the time difference and the video data collection and upload time of the motor vehicle inspection and supervision module.

3. The vehicle inspection and supervision system based on video recognition according to claim 2, characterized in that: Step a4 includes the following steps in detail: a41: Image noise reduction processing; Image frames are obtained based on the video data and denoted as: image data to be identified; noise reduction processing is performed on each frame of the image data to be identified. a42: The image data to be identified is scaled according to a preset size and then normalized; a43: Constructing a text detection model based on the DBnet model; The text detection model takes the image data to be recognized as input and outputs the coordinates of the text in the image data to be recognized. a44: The image data to be identified is fed into the text detection model to obtain the corresponding text coordinate position in the image data to be identified; a45: Based on the text coordinate position, each of the images to be identified is cropped to obtain a text region image; The text region image is scaled and normalized according to a specified size to obtain the text image to be recognized; a46: Constructing a text recognition model based on the CRNN model; The data for the text recognition model is the text image to be recognized, and the output is text characters. a47: Input the text image to be recognized into the text recognition model to obtain the corresponding text characters; The text characters are converted into time data according to a preset time format, thus obtaining the shooting time; a48: Subtract the shooting time from the video data acquisition and upload time to obtain the time difference.

4. The motor vehicle inspection and supervision system based on video recognition according to claim 1, characterized in that: The inspection business information management system and the motor vehicle inspection supervision module communicate via an external software interface for authorization; the inspection process information is stored using key encryption to prevent third parties from tampering with the process information.

5. The motor vehicle inspection and supervision system based on video recognition according to claim 1, characterized in that: The motor vehicle inspection and supervision module and the video recognition module exchange data using an encrypted signature method; asymmetric key pairs are generated for different algorithm providers; when the motor vehicle inspection and supervision module initiates a video detection request, the request parameters are signed with a private key, and the video recognition algorithm calculation module verifies the signature information using a public key; When the video recognition module writes the recognition results back to the motor vehicle inspection and supervision module, it uses an external software interface to apply for authorization.

6. The motor vehicle inspection and supervision system based on video recognition according to claim 3, characterized in that: The video recognition module includes: a video reading unit, a video recognition calculation unit, and a video result encapsulation and processing unit; After receiving the video recognition calculation request initiated by the system scheduling and control unit, the video reading unit reads the video stream stored in the audio and video acquisition management system through the video address and sends it to the video recognition calculation unit. The video recognition calculation unit performs frame extraction processing on the video stream, performs online video recognition according to the video type, performs calculations on the video according to the recognition project requirements, and sends the generated calculation results to the video result encapsulation processing unit. The video result encapsulation and processing unit encapsulates and packages the video recognition results in JSON format and uploads them to the video result analysis and processing unit in the motor vehicle inspection and supervision module for further processing.

7. The motor vehicle inspection and supervision system based on video recognition according to claim 1, characterized in that: The video storage unit supports two management methods for video surveillance equipment: Management Method 1: Each testing line is equipped with a workstation camera. The workstation program sends start and end signals to the corresponding workstation of the testing line, and the workstation notifies the workstation camera to take video and photos. After all videos are taken, they are stored in the WEB video server according to the preset retention period. Management Method 2: All video surveillance cameras are connected to a hard disk recorder, configured with corresponding ports, and the start and end times of each workstation are recorded. Based on the port and time, video data stored in the hard disk recorder is extracted and saved in the WEB video server according to the preset retention period. For video storage requirements of the global perspective, non-workstation video data of the corresponding perspective is obtained and stored in the hard disk recorder according to the preset retention period.

8. The motor vehicle inspection and supervision system based on video recognition according to claim 6, characterized in that: In the video recognition module, when the type of the motor vehicle to be confirmed is a flatbed truck and a flatbed trailer, the video recognition result includes: the height of the cargo box sideboards; In the video recognition calculation unit, the step of recognizing and calculating the height of the cargo box sideboard includes: b1: Construct a focus estimation model based on CenterNet; The input to the focus estimation model is the image data to be identified, and the output is the positions of the four corner points of the license plate of the vehicle to be identified and the positions of the four focal points of the rear panel of the vehicle. b2: Calculate the width and height of the license plate based on the four corner points, and record it as: license plate recognition dimensions; The width and height of the cargo box side panel are calculated based on the four focal points of the rear side panel of the vehicle and recorded as: cargo box side panel identification dimensions; b3: Obtain the standard dimensions of the license plate, denoted as: standard license plate dimensions; b4: Compare the standard license plate size with the license plate recognition size; Let w be the ratio of the width in the standard license plate dimensions to the width in the license plate recognition dimensions, and h be the ratio of the height. b5: Based on w and h, the actual width and height of the rear side panel of the vehicle are calculated by using the width and height in the dimensions of the cargo box side panel.

Images