Video-based vehicle driving track identification system and method
By adopting automatic exposure control and image enhancement algorithm in high-definition cameras, combined with interpolation method and multi-source data fusion, the problem of vehicle driving trajectory recognition system detection and occlusion in different lighting conditions and dense traffic scenes is solved, achieving more accurate and continuous vehicle track identification.
Patent Information
- Application Number
- CN202411241299.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-03
AI Technical Summary
Existing video-based vehicle driving trajectory recognition systems have problems with vehicle detection and identification accuracy under different lighting conditions and in dense traffic scenarios, especially because information loss caused by light changes and vehicle occlusion affects the continuity and accuracy of the trajectory. .
Image enhancement algorithms such as automatic exposure control, histogram equalization and Gamma correction of high-definition cameras are used to process image quality problems, and vehicle position prediction and occlusion compensation are carried out through interpolation and multi-source data fusion, and images are improved by combining filtering algorithms and morphological operations. Mass and trajectory integrity.
Improve the accuracy of vehicle detection under different lighting conditions, reduce trajectory mutations caused by occlusion, enhance the continuity and integrity of the trajectory, and ensure the accuracy and reliability of the vehicle's driving trajectory.
Smart Images

Figure CN119068440A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of road traffic technology, and in particular to a vehicle driving trajectory recognition system and method based on video. Background Art
[0002] The video-based vehicle trajectory recognition system uses cameras installed along the road to capture video images, and through advanced image processing algorithms and deep learning technology, it automatically detects and identifies the vehicle's location, speed, direction of travel and other information, and constructs the vehicle's trajectory. This trajectory information is then transmitted to the traffic management center for further analysis and decision-making, such as traffic flow management, accident warning, violation behavior monitoring, etc. After checking the publication number: CN114998815B, a method and system for identifying and tracking traffic vehicles based on video analysis is disclosed. This technology discloses "a method and system for identifying and tracking traffic vehicles based on video analysis, the method comprising: step 1, extracting a first video and a second video, the first video and the second video having a temporal and spatial correlation; step 2, performing vehicle identification on the first video; step 3, performing vehicle identification on the second video; step 4, marking the successfully identified vehicles with the same license plate on the map, connecting the marked points on the map according to the temporal and spatial correlation of the first video and the second video, obtaining the vehicle driving trajectory, and completing vehicle tracking according to the vehicle driving trajectory" and other technical solutions, with "the efficiency of vehicle detection, identification and tracking is improved by integrating the improved swin transformer deep learning model and the AlexNet model trained under the MapReduce framework" and other technical effects; Although the existing video-based vehicle driving trajectory recognition system has made significant progress in practical applications, it still has some shortcomings. These shortcomings are mainly reflected in: under different lighting conditions (such as sunny days, cloudy days, nights, etc.), the quality of video images will change, affecting the accuracy of vehicle detection and recognition; and in traffic scenes, vehicles may occlude each other, especially in dense traffic flows, the occlusion problem is more prominent, which will cause some vehicle information to be lost, affecting the continuity and accuracy of the trajectory. Summary of the invention
[0003] In view of the deficiencies in the prior art, the present invention provides a video-based vehicle driving trajectory recognition system and method, which has the characteristics of adaptive processing of lighting changes, vehicle trajectory prediction and road conditions when facing interference from environmental factors.
[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: a vehicle driving trajectory recognition system based on video, including a vehicle driving trajectory recognition system, the vehicle driving trajectory recognition system including a video acquisition module, a digital image preprocessing module, a vehicle recognition and tracking module, a trajectory generation and visualization module, a data analysis and application module and a control display panel; The video acquisition module comprises a high-definition camera, a processing unit and a driver unit. The output end of the high-definition camera is connected to the input end of the processing unit, and the output end of the processing unit is connected to the input end of the driver unit.
[0005] Preferably, the output end of the driver unit is connected to the input end of the digital image preprocessing module, and the output end of the digital image preprocessing module is connected to the input end of the vehicle identification and tracking module, the output end of the vehicle identification and tracking module is connected to the input end of the trajectory generation and visualization module, the output end of the trajectory generation and visualization module is connected to the input end of the data analysis and application module and the control display panel, and a bidirectional connection is achieved between the data analysis and application module and the control display panel.
[0006] The present invention also discloses a method for a vehicle driving trajectory recognition system based on video, which specifically comprises the following steps: S1. Digital image preprocessing: The video acquisition module is the source of the video signal and is responsible for capturing the driving conditions of vehicles on the road. The digital image preprocessing module decodes the acquired video stream into continuous image frames, eliminates or reduces the noise in the image, improves the visual effect of the image, enhances the image contrast, highlights the vehicle features, and converts the image into a binary image to simplify the image information. S2. Vehicle recognition and tracking: The vehicle recognition and tracking module matches and recognizes the features of the edge, corner point and color histogram of the vehicle through the target detection algorithm. The detected vehicle object will be passed to the target tracking algorithm to track the detected vehicle object in continuous video frames. S3, trajectory generation and visualization: The trajectory generation and visualization module receives the vehicle coordinates and timestamp location information output by the vehicle identification and tracking module, and constructs the continuous driving trajectory of the vehicle based on the preprocessed vehicle location information. It then extracts key features such as trajectory length, driving speed, acceleration and steering angle from the generated trajectory data, and stores the generated trajectory data in the database for subsequent query, analysis and reuse; S4. Data analysis and application: The data analysis and application module cleans, denoises, and processes missing values on the trajectory data, extracts features for subsequent analysis of speed, acceleration, steering angle, and driving distance, performs statistical analysis on the trajectory data, calculates statistical indicators such as mean, standard deviation, maximum, and minimum, and uses cluster analysis, classification algorithms, regression models, and time series analysis to deeply mine the trajectory data and discover potential patterns and rules.
[0007] Preferably, in step S1, after the image is binarized, edge information in the image is extracted, and then the image is subjected to noise elimination, hole filling and connection break processing.
[0008] Preferably, in step S2, when using the target tracking algorithm, by comparing the vehicle position, speed, acceleration and other information between consecutive frames, the association between vehicles is established to achieve cross-frame tracking, and in the case where the vehicle is occluded, the prediction model is used or the context information is combined to perform occlusion compensation to maintain the continuity of tracking, and the tracked vehicle position information is arranged in chronological order to form the vehicle's driving trajectory, and smoothing is performed to eliminate jitter.
[0009] Preferably, in step S3, the trajectory data is converted into a graphical display form by a graphics rendering engine, and the trajectory data is displayed on the user interface in the form of lines and icons with the cooperation of a control display panel.
[0010] Preferably, in step S4, the trajectory data analysis results are used to monitor and manage traffic flow, congestion, violations, etc., and an optimization plan is proposed. Then, the trajectory data and urban planning requirements are combined to provide decision support for urban infrastructure construction.
[0011] Preferably, in step S4, the trajectory data is used to quickly locate the accident scene, analyze the cause of the accident, and formulate an emergency response plan; then, based on the trajectory data, consumer behavior patterns and commercial activity hotspots are analyzed, and business intelligence services are provided to merchants.
[0012] The present invention provides a vehicle driving trajectory recognition system and method based on video. Compared with the prior art, it has the following beneficial effects: 1. Through the built-in automatic exposure control function of the high-definition camera, the exposure parameters of the camera are dynamically adjusted to adapt to scenes under different lighting conditions; and image enhancement algorithms such as histogram equalization and gamma correction are used to improve the overall brightness and contrast of the image, so that the vehicle target can be clearly seen under different lighting conditions; for severe weather conditions such as heavy rain, heavy snow, and heavy fog, specific filtering algorithms (such as median filtering, Gaussian filtering, etc.) are used to remove noise and interference in the image and improve image quality. Image processing algorithms (such as background subtraction, optical flow method, etc.) are used to detect and suppress light spots and reflections generated by road reflections to reduce interference with vehicle detection.
[0013] 2. When it is predicted that the vehicle may be occluded, the interpolation method (such as linear interpolation, polynomial interpolation, etc.) is used to estimate the position of the vehicle. This method makes a reasonable guess about the position during the occlusion based on the vehicle's historical motion trajectory and predicted motion state; and the trajectory before and after the occlusion is smoothed to reduce the trajectory mutation caused by the occlusion. Trajectory smoothing can be achieved through various filtering algorithms, such as Kalman filtering, median filtering, etc.; at the same time, multiple groups of high-definition cameras are set up to compensate for the missing trajectory caused by occlusion through multi-source data fusion, and the information complementarity between different data sources can improve the integrity and accuracy of the trajectory.
[0014] 3. Use mean filtering, median filtering, Gaussian filtering and other algorithms to smooth the image and reduce noise interference; use histogram equalization, contrast stretching, sharpening and other algorithms to process the image to make the vehicle outline clearer; then set the threshold, set the pixels in the image greater than the threshold to white, and the pixels less than the threshold to black, so as to obtain a binary image; then use edge detection algorithms such as Sobel operator and Canny operator to process the image and identify the edge lines in the image; finally, use morphological operation algorithms such as dilation, erosion, opening operation and closing operation to perform local processing on the image to improve image quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 is a functional block diagram of the system of the present invention; Figure 2 It is a principle block diagram of the video acquisition module in the present invention; Figure 3 It is a flow chart of the method for digital image preprocessing in the present invention; Figure 4 A flow chart of the method for vehicle identification and tracking in the present invention; Figure 5 It is a flow chart of the method of trajectory generation and visualization module in the present invention; Figure 6 It is a method flow chart of the data analysis and application module in the present invention.
[0016] In the figure: 1. Vehicle driving trajectory recognition system; 11. Video acquisition module; 111. High-definition camera; 112. Processing unit; 113. Driver unit; 12. Digital image preprocessing module; 13. Vehicle recognition and tracking module; 14. Trajectory generation and visualization module; 15. Data analysis and application module; 16. Control display panel. DETAILED DESCRIPTION
[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0018] See also Figure 1-6 , the present invention provides a vehicle driving trajectory recognition system and method technical solution based on video: a vehicle driving trajectory recognition system based on video, comprising a vehicle driving trajectory recognition system 1, the vehicle driving trajectory recognition system 1 comprises a video acquisition module 11, a digital image preprocessing module 12, a vehicle recognition and tracking module 13, a trajectory generation and visualization module 14, a data analysis and application module 15 and a control display panel 16; The video acquisition module 11 includes a high-definition camera 111 , a processing unit 112 and a driver unit 113 . The output end of the high-definition camera 111 is connected to the input end of the processing unit 112 , and the output end of the processing unit 112 is connected to the input end of the driver unit 113 .
[0019] In this implementation, the high-definition camera 111 has a built-in automatic exposure control function that dynamically adjusts the exposure parameters of the camera to adapt to scenes under different lighting conditions; and uses image enhancement algorithms such as histogram equalization and gamma correction to improve the overall brightness and contrast of the image, so that the vehicle target can be clearly seen under different lighting conditions; for severe weather conditions such as heavy rain, heavy snow, and heavy fog, specific filtering algorithms (such as median filtering, Gaussian filtering, etc.) are used to remove noise and interference in the image and improve image quality, and image processing algorithms (such as background subtraction, optical flow method, etc.) are used to detect and suppress light spots and reflections generated by road surface reflections to reduce interference with vehicle detection.
[0020] Specifically, the output end of the driver unit 113 is connected to the input end of the digital image preprocessing module 12, and the output end of the digital image preprocessing module 12 is connected to the input end of the vehicle identification and tracking module 13, the output end of the vehicle identification and tracking module 13 is connected to the input end of the trajectory generation and visualization module 14, the output end of the trajectory generation and visualization module 14 is connected to the input end of the data analysis and application module 15 and the control display panel 16, and a bidirectional connection is achieved between the data analysis and application module 15 and the control display panel 16.
[0021] The present invention also discloses a method for a vehicle driving trajectory recognition system based on video, which specifically comprises the following steps: S1, digital image preprocessing: the video acquisition module 11 is the source of the video signal and is responsible for capturing the driving conditions of vehicles on the road, and decodes the acquired video stream into continuous image frames through the digital image preprocessing module 12, then eliminates or reduces the noise in the image, improves the visual effect of the image, enhances the image contrast, highlights the vehicle features, and then converts the image into a binary image to simplify the image information; S2, vehicle identification and tracking: The vehicle identification and tracking module 13 matches and identifies the features of the edge, corner point and color histogram of the vehicle through the target detection algorithm. The detected vehicle object will be passed to the target tracking algorithm to track the detected vehicle object in the continuous video frames; S3, trajectory generation and visualization: The trajectory generation and visualization module 14 receives the vehicle coordinates and timestamp location information output by the vehicle identification and tracking module 13, and constructs the continuous driving trajectory of the vehicle according to the pre-processed vehicle location information, and then extracts the key features of the trajectory length, driving speed, acceleration and steering angle from the generated trajectory data, and stores the generated trajectory data in the database for subsequent query, analysis and reuse; S4. Data analysis and application: The trajectory data is cleaned, denoised, and missing values are processed through the data analysis and application module 15, and features such as speed, acceleration, steering angle, and driving distance for subsequent analysis are extracted from the trajectory data. The trajectory data is statistically analyzed, and statistical indicators such as mean, standard deviation, maximum, and minimum are calculated. Cluster analysis, classification algorithms, regression models, and time series analysis methods are used to deeply mine the trajectory data to discover potential patterns and rules.
[0022] In this embodiment, when it is predicted that the vehicle may be occluded, the interpolation method (such as linear interpolation, polynomial interpolation, etc.) is used to estimate the position of the vehicle. This method makes a reasonable guess about the position during the occlusion based on the historical motion trajectory of the vehicle and the predicted motion state; and the trajectory before and after the occlusion is smoothed to reduce the trajectory mutation caused by the occlusion. Trajectory smoothing can be achieved through various filtering algorithms, such as Kalman filtering, median filtering, etc.; at the same time, multiple groups of high-definition cameras 111 are set to compensate for the trajectory loss caused by occlusion through multi-source data fusion, and the information complementarity between different data sources can improve the integrity and accuracy of the trajectory.
[0023] Specifically, in step S1, after the image is binarized, edge information in the image is extracted, and then the image is subjected to noise elimination, hole filling and connection break processing.
[0024] In this embodiment, algorithms such as mean filtering, median filtering, and Gaussian filtering are used to smooth the image to reduce noise interference; and algorithms such as histogram equalization, contrast stretching, and sharpening are used to process the image to make the vehicle outline clearer; then by setting a threshold, pixels in the image that are greater than the threshold are set to white, and pixels that are less than the threshold are set to black, thereby obtaining a binary image; then edge detection algorithms such as the Sobel operator and the Canny operator are used to process the image to identify edge lines in the image; finally, morphological operation algorithms such as dilation, erosion, opening operation, and closing operation are used to locally process the image to improve image quality.
[0025] Specifically, in step S2, when the target tracking algorithm is used, the association between vehicles is established by comparing the vehicle position, speed, acceleration and other information between consecutive frames to achieve cross-frame tracking. In the case where the vehicle is occluded, a prediction model is used or context information is combined to perform occlusion compensation to maintain tracking continuity. The tracked vehicle position information is arranged in chronological order to form the vehicle's driving trajectory, and smoothing is performed to eliminate jitter.
[0026] Specifically, in step S3 , the trajectory data is converted into a graphical display form by a graphics rendering engine, and the trajectory data is displayed on the user interface in the form of lines and icons with the cooperation of the control display panel 16 .
[0027] In this embodiment, the accuracy and consistency of the data are ensured by performing preprocessing operations such as time synchronization and coordinate conversion on the received data; and then an interpolation algorithm (such as linear interpolation, spline interpolation, etc.) is used to fill the gaps between the position information to ensure the smoothness and continuity of the trajectory.
[0028] Specifically, in step S4, the trajectory data analysis results are used to monitor and manage traffic flow, congestion, violations, etc., and an optimization plan is proposed. Then, the trajectory data and urban planning requirements are combined to provide decision support for urban infrastructure construction.
[0029] Specifically, in step S4, the trajectory data is used to quickly locate the accident scene, analyze the cause of the accident, and formulate an emergency response plan; then, based on the trajectory data, consumer behavior patterns and commercial activity hotspots are analyzed, and business intelligence services are provided to merchants.
[0030] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0031] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A vehicle driving trajectory recognition system based on video, characterized in that: The vehicle driving trajectory recognition system (1) comprises a video acquisition module (11), a digital image preprocessing module (12), a vehicle recognition and tracking module (13), a trajectory generation and visualization module (14), a data analysis and application module (15) and a control display panel (16); The video acquisition module (11) comprises a high-definition camera (111), a processing unit (112) and a driver unit (113); the output end of the high-definition camera (111) is connected to the input end of the processing unit (112), and the output end of the processing unit (112) is connected to the input end of the driver unit (113).
2. The vehicle driving trajectory recognition system based on video according to claim 1, characterized in that: The output end of the driver unit (113) is connected to the input end of the digital image preprocessing module (12), and the output end of the digital image preprocessing module (12) is connected to the input end of the vehicle identification and tracking module (13), the output end of the vehicle identification and tracking module (13) is connected to the input end of the trajectory generation and visualization module (14), the output end of the trajectory generation and visualization module (14) is connected to the input end of the data analysis and application module (15) and the control display panel (16), and a bidirectional connection is achieved between the data analysis and application module (15) and the control display panel (16).
3. A method for a vehicle driving trajectory recognition system based on video according to any one of claims 1-2, characterized in that: The specific steps include: S1. Digital image preprocessing: The video acquisition module (11) is the source of the video signal and is responsible for capturing the driving conditions of vehicles on the road. The acquired video stream is decoded into continuous image frames through the digital image preprocessing module (12), and then the noise in the image is eliminated or reduced, and the visual effect of the image is improved, the image contrast is enhanced, the vehicle features are highlighted, and the image is converted into a binary image to simplify the image information. S2, vehicle identification and tracking: The vehicle identification and tracking module (13) matches and identifies the features of the edge, corner point and color histogram of the vehicle through the target detection algorithm. The detected vehicle object will be passed to the target tracking algorithm to track the detected vehicle object in the continuous video frames; S3, trajectory generation and visualization: The trajectory generation and visualization module (14) receives the vehicle coordinates and timestamp location information output by the vehicle identification and tracking module (13), and constructs the continuous driving trajectory of the vehicle based on the pre-processed vehicle location information, and then extracts key features such as trajectory length, driving speed, acceleration and steering angle from the generated trajectory data, and stores the generated trajectory data in a database for subsequent query, analysis and reuse; S4. Data analysis and application: The data analysis and application module (15) cleans, removes noise, and processes missing values on the trajectory data. It also extracts features such as speed, acceleration, steering angle, and driving distance for subsequent analysis. It performs statistical analysis on the trajectory data, calculates statistical indicators such as mean, standard deviation, maximum, and minimum, and uses cluster analysis, classification algorithms, regression models, and time series analysis to conduct in-depth mining of the trajectory data and discover potential patterns and rules.
4. The method of a vehicle driving trajectory recognition system based on video according to claim 3 is characterized in that: In the step S1, after the image is binarized, edge information in the image is extracted, and then the image is subjected to noise elimination, hole filling and connection break processing.
5. The method of a vehicle driving trajectory recognition system based on video according to claim 3, characterized in that: In step S2, when the target tracking algorithm is used, the association between vehicles is established by comparing the vehicle position, speed, acceleration and other information between consecutive frames to achieve cross-frame tracking. In the case where the vehicle is occluded, the prediction model is used or the context information is combined to perform occlusion compensation to maintain the continuity of tracking. The tracked vehicle position information is arranged in chronological order to form the vehicle's driving trajectory, and smoothing is performed to eliminate jitter.
6. The method of a vehicle driving trajectory recognition system based on video according to claim 3, characterized in that: In step S3, the trajectory data is converted into a graphical display form by a graphics rendering engine, and the trajectory data is displayed on the user interface in the form of lines and icons in cooperation with the control display panel (16).
7. The method of a vehicle driving trajectory recognition system based on video according to claim 3, characterized in that: In step S4, the trajectory data analysis results are used to monitor and manage traffic flow, congestion, violations, etc., and an optimization plan is proposed. Combined with the trajectory data and urban planning requirements, decision support is provided for urban infrastructure construction.
8. The method of a vehicle driving trajectory recognition system based on video according to claim 3, characterized in that: In step S4, the trajectory data is used to quickly locate the accident scene, analyze the cause of the accident, and formulate an emergency response plan; then, based on the trajectory data, consumer behavior patterns and commercial activity hotspots are analyzed, and business intelligence services are provided to merchants.
Citation Information
Patent Citations
A traffic vehicle identification and tracking method and system based on video analytics
CN114998815B
