Vehicle violation supervision system based on video analysis and application
Through a vehicle violation supervision system based on video analysis, using video acquisition, network transmission and AI management subsystems, the shortcomings of the existing system in vehicle details identification and data integration are solved, and efficient vehicle information capture and violation identification are achieved.
Patent Information
- Application Number
- CN202411931728.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-11
AI Technical Summary
The existing electronic monitoring systems have shortcomings in vehicle detail identification and complex road conditions monitoring, and have failed to achieve efficient data integration and deep integration with other important transportation systems.
The vehicle violation supervision system based on video analysis is adopted, including the roadside video acquisition subsystem, network transmission subsystem and AI-driven management subsystem. The capture, processing and identification of vehicle information is achieved through embedded cameras, AI processors, convolutional neural network models and efficient data transmission equipment.
It realizes effective capture and identification of vehicle circulation information, driver facial features and license plate numbers, improves the high-precision detection capabilities of vehicle locations, and ensures accurate identification and management of vehicle violations.
Smart Images
Figure CN119811075A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traffic monitoring, and in particular to a vehicle violation monitoring system based on video analysis and its application. Background Art
[0002] With the booming global economy and rapid technological progress, the importance of electronic monitoring systems in the field of transportation has become increasingly prominent. This system has evolved from simple functions such as basic violation capture to a comprehensive platform with diverse advanced functions. Modern electronic monitoring systems not only include standard red light running and traffic rule violation capture functions, but also integrate advanced functions closely related to urban traffic management such as traffic statistics and traffic congestion monitoring, contributing to the construction of smart cities.
[0003] Nevertheless, the rapid development of electronic monitoring systems has also brought new challenges and problems, especially in terms of technical implementation and system planning. These problems not only restrict the potential efficient operation of the system, but also constitute an obstacle to the efforts of traffic management departments to improve road safety and efficiency. For example, although modern electronic monitoring systems can detect a variety of violations, they are still insufficient in identifying detailed features of vehicles (such as vehicle brands and models) and monitoring complex road conditions (such as real-time traffic flow and lane usage). In addition, most of the current systems have not yet achieved efficient data integration and deep integration with other important traffic systems (such as traffic signal control systems and traffic information service systems);
[0004] In order to solve the above problems, a vehicle violation supervision system and application based on video analysis is proposed. Summary of the invention
[0005] According to the above existing technical problems, the present invention provides a vehicle violation supervision system and application based on video analysis, which is characterized in that it is mainly composed of three parts: a video acquisition subsystem at the road end, a network transmission subsystem, and an AI-driven management subsystem. The video acquisition subsystem at the road end mainly includes an image acquisition module integrating an embedded camera and an AI processor, an auxiliary light source, an edge computing device, and a network transmission device, which is specifically used to capture road images and vehicle information. The AI management subsystem is responsible for processing the records obtained by capturing, including the association and synthesis of records, breakpoint data transmission, image and video retrieval, image synthesis, and black and white list management. The network transmission subsystem is composed of an Ethernet switch and an optical fiber transmission device, which ensures efficient and stable data transmission from the road end to the AI management subsystem. The AI management subsystem also includes a background processing server, a data storage server, and a query terminal device. These parts together ensure the efficiency and accuracy of the entire system in capturing and processing violation information.
[0006] The network transmission subsystem of the system consists of an Ethernet switch and a fiber optic transmission device. The network transmission subsystem gives priority to fiber optic transmission. If the fiber optic line cannot be covered, it will turn to a wireless access solution: in this case, 5G wireless transmission technology is used for data communication;
[0007] In this intelligent traffic violation automatic identification method and system, the vehicle-side equipment mainly consists of three units: image input, image processing and convolutional neural network model. The image input unit is responsible for receiving the image signal captured by the roadside camera and transmitting it to the image processing unit. Subsequently, the image processing unit performs denoising on these received images to improve their quality. The processed images are then sent to the convolutional neural network model unit, where the system classifies the moving objects in the image to accurately identify the relevant information of the vehicle;
[0008] In the intelligent traffic violation automatic identification method and system, the core function of the image processing unit is to decompose the original two-dimensional image signal into a series of components with different frequencies and a residual part. Specifically, the original image data I is split into a plurality of frequency components F and a residual R. This decomposition process converts the image into a plurality of components with different frequencies.
[0009] To further process these components, the system uses a two-dimensional Hilbert-Huang transform (HHT), which involves convolving each frequency component with the HHT operator H in order to extract key information from the image;
[0010] After completing the convolution operation, the system recombines the processed image components with the remaining residue R. The formula used in this reconstruction process involves combining each transformed eigenmode function EMk and adding the residue R to generate a denoised image. In short, this process aims to extract clearer and more accurate information from the original image through advanced signal processing technology, providing high-quality input data for subsequent vehicle recognition and classification.
[0011] The processed image is fed into the convolutional neural network (CNN) model. The model structure includes six convolutional layers, four fully connected layers, one attention layer, and one output layer. Its parameter setting and operation process can be summarized in the following steps:
[0012] S1 (image input): The model first receives a three-channel image of size 640×640 pixels;
[0013] S2 (first convolutional layer operation): convolve the input image using 128 9×9 kernels with a stride of 5 pixels, and then apply a 3×3 kernel for maximum pooling.
[0014] S3 (second convolutional layer operation): 256 7×7 convolution kernels are used to process the output of the first convolutional layer, and then the maximum pooling is performed using the 3×3 kernel.
[0015] S4 (the third convolutional layer operation): 512 5×5 convolution kernels are applied to convolve the output of the second layer, followed by a 3×3 kernel maximum pooling.
[0016] S5 (create and run a parallel branch): treat the operations of steps 2 to 4 as a branch and create a second branch with the same structure;
[0017] S6 (attention layer integration): merge the outputs of the two parallel branches and then send them to the attention layer;
[0018] S7 (fully connected layer processing): The output of the attention layer first enters the first fully connected layer, which has 2048 neurons;
[0019] S8 (second fully connected layer): The data then flows into the second fully connected layer configured with 4096 neurons;
[0020] S9 (create a second fully connected layer branch): create a fully connected layer of the second branch according to the structure of steps 7 and 8;
[0021] S10 (output layer summary): Finally, the outputs of the two fully connected layer branches are merged and sent to the output layer for final processing;
[0022] In this convolutional neural network model unit, sample collection involves obtaining data from Internet resources and real-world scenarios, and then dividing these data into positive samples and negative samples. Positive samples mainly include vehicle images taken from various angles, while negative samples include non-vehicle images taken from the Internet or in the field. In order to ensure the consistency of data processing, all these samples are adjusted to a unified 640×640 pixel size for standardization.
[0023] The workflow of this intelligent traffic violation automatic identification system can be summarized into the following core steps:
[0024] T1 (Video Capture): The HD camera capture host first captures the real-time video stream;
[0025] T2 (image processing and feature analysis): The system uses a multi-layer convolutional neural network to deeply analyze the images in the video stream, extract the key features of the images, and classify and estimate their locations;
[0026] T3 (target detection): The system identifies the target area and provides the exact location and size information (including width and height) of the area;
[0027] T4 (Vehicle Positioning and Triggering): Analyze the specific position of the vehicle to determine whether the vehicle has reached the set trigger line. If the vehicle has not reached the trigger line, the system will continue to analyze subsequent image frames. If the vehicle touches or crosses the trigger line, the system will send a signal and may proceed to the next step of processing, such as image capture or other operations.
[0028] Beneficial effects of the present invention:
[0029] The present invention can effectively capture vehicle circulation information and driver's facial features, and automatically identify license plate numbers. The design of the system emphasizes the stability of the equipment and the simplicity of the structure, making installation and maintenance easier. In addition, the system adopts vehicle detection technology based on deep neural networks to ensure high-precision detection of vehicle positions, thereby accurately identifying various vehicle violations. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a schematic diagram of the overall architecture of the vehicle violation supervision system and application based on video analysis of the present invention;
[0031] Figure 2 It is a schematic diagram of violation logic judgment of the present invention;
[0032] Figure 3 Schematic diagram of the convolutional neural network structure of the present invention. DETAILED DESCRIPTION
[0033] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.
[0035] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0036] Example 1
[0037] The present invention provides a vehicle violation supervision system and application based on video analysis, which is characterized in that it is composed of a road-side video acquisition subsystem, a network transmission subsystem and an AI management subsystem. The road-side video acquisition subsystem includes an image acquisition module, an auxiliary light source, an edge computing device and a network transmission device. The image acquisition module is equipped with an embedded camera with an integrated structure and a built-in AI processor, and is also equipped with vehicle-side equipment. The AI management subsystem is responsible for processing the association and synthesis of captured records, breakpoint continuation, image and video retrieval, image synthesis and management of black and white lists. The network transmission subsystem of the system is composed of an Ethernet switch and optical fiber transmission equipment, and the AI management subsystem includes a background processing server, a data storage server and a query terminal device. This architecture ensures effective data exchange and communication from the road-side video acquisition subsystem to the AI management subsystem.
[0038] In this system, the image input unit is responsible for receiving the image signal from the capture unit and passing it to the image processing unit. In the image processing unit, the received image is first denoised and then sent to the convolutional neural network model unit. In this unit, the system recognizes and classifies the moving objects in the image to accurately determine the relevant information of the vehicle.
[0039] Video image capture: Use a high-frame-rate high-definition camera to continuously record and collect real-time information about vehicles passing through the checkpoint and vehicle violations. In the intelligent transportation system, vehicle capture is usually performed immediately after the vehicle is detected. The frame rate directly affects the accuracy of the capture position. A higher frame rate means a shorter time interval between frames, which shortens the delay distance of the vehicle during capture, greatly improving the performance of vehicle positioning. The intelligent transportation system has extremely high requirements for recognition rate, and the ideal position interval for capture is usually very limited, about 10 meters. When vehicles of the same speed pass by, a higher frame rate provides more image frames for analysis and recognition, thereby significantly improving the accuracy of recognition;
[0040] The violation identification business process includes:
[0041] Vehicle detection: By inputting an image into a trained vehicle detection neural network model, the model will output relevant information such as the specific location, width, and height of the vehicle after processing;
[0042] License plate detection: The vehicle image identified by vehicle detection is input into another trained license plate detection neural network model, which will provide information such as the specific location, width and height of the license plate after processing;
[0043] License plate recognition: Next, the detected license plate image is input into a specialized license plate information recognition neural network model, which outputs the specific information on the license plate after calculation;
[0044] Result display: Finally, the system displays the results of license plate recognition in text form;
[0045] The core function of the image processing unit is to decompose the original two-dimensional image signal into a series of components with different frequencies and a residual part. Specifically, the original image data I is split into multiple frequency components F and a residual R. This decomposition process converts the image into several components with different frequencies.
[0046] To further process these components, the system uses a two-dimensional Hilbert-Huang transform (HHT), which involves convolving each frequency component with the HHT operator H in order to extract key information from the image;
[0047] After completing the convolution operation, the system recombines the processed image components with the remaining residue R. The formula used in this reconstruction process involves combining each transformed eigenmode function EMk and adding the residue R to generate a denoised image. In short, this process aims to extract clearer and more accurate information from the original image through advanced signal processing technology, providing high-quality input data for subsequent vehicle recognition and classification.
[0048] The processed image is fed into the convolutional neural network (CNN) model. The model structure includes six convolutional layers, four fully connected layers, one attention layer, and one output layer. Its parameter setting and operation process can be summarized in the following steps:
[0049] S1 (image input): The model first receives a three-channel image of size 640×640 pixels;
[0050] S2 (first convolutional layer operation): convolve the input image using 128 9×9 kernels with a stride of 5 pixels, and then apply a 3×3 kernel for maximum pooling.
[0051] S3 (second convolutional layer operation): 256 7×7 convolution kernels are used to process the output of the first convolutional layer, and then the maximum pooling is performed using the 3×3 kernel.
[0052] S4 (the third convolutional layer operation): 512 5×5 convolution kernels are applied to convolve the output of the second layer, followed by a 3×3 kernel maximum pooling.
[0053] S5 (create and run a parallel branch): treat the operations of steps 2 to 4 as a branch and create a second branch with the same structure;
[0054] S6 (attention layer integration): merge the outputs of the two parallel branches and then send them to the attention layer;
[0055] S7 (fully connected layer processing): The output of the attention layer first enters the first fully connected layer, which has 2048 neurons;
[0056] S8 (second fully connected layer): The data then flows into the second fully connected layer configured with 4096 neurons;
[0057] S9 (create a second fully connected layer branch): create a fully connected layer of the second branch according to the structure of steps 7 and 8;
[0058] S10 (output layer summary): Finally, the outputs of the two fully connected layer branches are merged and sent to the output layer for final processing;
[0059] In this convolutional neural network model unit, sample collection involves obtaining data from Internet resources and real-world scenarios, and then dividing these data into positive samples and negative samples. Positive samples mainly include vehicle images taken from various angles, while negative samples include non-vehicle images taken from the Internet or in the field. In order to ensure the consistency of data processing, all these samples are adjusted to a unified 640×640 pixel size for standardization.
[0060] The workflow of this intelligent traffic violation automatic identification system can be summarized into the following core steps:
[0061] T1 (Video Capture): The HD camera capture host first captures the real-time video stream;
[0062] T2 (image processing and feature analysis): The system uses a multi-layer convolutional neural network to deeply analyze the images in the video stream, extract the key features of the images, and classify and estimate their locations;
[0063] T3 (target detection): The system identifies the target area and provides the exact location and size information (including width and height) of the area;
[0064] T4 (Vehicle Positioning and Triggering): Analyze the specific position of the vehicle to determine whether the vehicle has reached the set trigger line. If the vehicle has not reached the trigger line, the system will continue to analyze subsequent image frames. If the vehicle touches or crosses the trigger line, the system will send a signal and may proceed to the next step of processing, such as image capture or other operations.
[0065] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. The various components mentioned in the present invention are common technologies in the existing field. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only to illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.
Claims
1. The vehicle violation supervision system based on video analysis is characterized by: It consists of a road-side video acquisition subsystem, a network transmission subsystem and an AI management subsystem. The road-side video acquisition subsystem includes an image acquisition module, an auxiliary light source, an edge computing device and a network transmission device. The image acquisition module is equipped with an integrated embedded camera and a built-in AI processor, and is also equipped with vehicle-side equipment. The network transmission subsystem of the system consists of an Ethernet switch and optical fiber transmission equipment. The AI management subsystem is responsible for processing the record association and synthesis of electric police capture and checkpoint capture, breakpoint resumption, image and video retrieval, image synthesis and black and white list management, and includes a background processing server, a data storage server and a query terminal device.
2. The vehicle violation supervision system based on video analysis according to claim 1 is characterized in that: The vehicle-side equipment consists of three parts: an image input unit, an image processing unit and a convolutional neural network model unit. The image input unit is responsible for receiving the image signal collected by the capture unit and transmitting it to the image processing unit. The image processing unit denoises the received image and then sends the processed image to the convolutional neural network model unit. In the convolutional neural network model unit, the dynamic target area in the image is classified and processed to identify and confirm the vehicle information.
3. The vehicle violation supervision system based on video analysis according to claim 2 is characterized in that: The image processing unit decomposes the received original two-dimensional image signal into several components F of different frequencies and a residue R. The decomposed image is composed of components of different frequencies. The decomposed different frequency components are processed using a two-dimensional Hilbert-Huang transform. Each frequency component is convolved with an HHT operator H to obtain a convolution result. Then, the convolution result is recombined with the residue R obtained after the decomposition of the original image to reconstruct the image.
4. The vehicle violation supervision system based on video analysis according to claim 3 is characterized in that: The convolutional neural network model unit consists of 6 convolutional layers, 4 fully connected layers, 1 attention layer and 1 output layer. The model parameter settings and operation flow are as follows: S1, input 640×640 three-channel image; S2, the first convolutional layer uses 128 9×9 convolution kernels with a stride of 5 pixels to perform convolution operations on the input image, and then uses 3×3 kernels to perform maximum pooling on the results; S3, the second convolution layer uses 256 convolution kernels of size 7×7 to perform convolution operation on the feature map output by the first layer, and then uses 3×3 kernels to perform maximum pooling on the result; S4, the third convolutional layer uses 512 convolution kernels of size 5×5 to perform convolution operation on the feature map output by the previous layer, and then uses 3×3 kernels to perform maximum pooling on the result; S5, using S2-S4 as the first branch, create the second branch with the same structure; S6, superimpose the first branch and the second branch and send them to the attention layer; S7, send the result of S6 to the first branch of the fully connected layer, the first fully connected layer is 2048 dimensions; S8, the first fully connected layer is 4096 layers; S9, create the second branch of the fully connected layer according to the structure of S7-S8; S10, superimpose the first and second fully connected layer branches and send them to the output layer.
5. The vehicle violation supervision system based on video analysis according to claim 4 is characterized in that: The sample collection process of the convolutional neural network model unit includes data obtained from online resources and real life. These data are classified into positive samples and negative samples. The positive samples are composed of vehicle images captured from different angles, and the negative samples are derived from other images taken from the Internet and on-site. Both positive and negative samples are processed into a size of 640×640 pixels.
6. The application of the vehicle violation supervision system based on video analysis according to claims 1-5 is characterized in that: Its operation process includes the following key steps: T1. Video stream acquisition: First, the HD camera capture host captures the real-time video stream; T2, Image feature extraction and analysis: Use multi-layer convolutional neural networks to extract key features from images, and classify and estimate the locations of these features; T3, target area determination: output the specific location and size (width and height) information of the area identified as the target; T4, vehicle position determination: Analyze the specific position of the vehicle to determine whether it has reached the preset trigger line. If the vehicle has not reached the trigger line, the system continues to process the next frame of image; if the vehicle has reached the trigger line, the system sends a trigger signal and may perform further processing or capture.