Vehicle trajectory tracking method and system based on video analysis
By analyzing and processing vehicle traffic videos on multiple road sections, determining the location adjacent relationship and vehicle driving order, and filling the video gap, the problem of low adaptability and accuracy of vehicle trajectory tracking is solved, and the integrity of vehicle trajectory and intelligent traffic is achieved. Improve management efficiency.
Patent Information
- Application Number
- CN202411448076.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2044-10-17
AI Technical Summary
In the prior art, the adaptability and accuracy of vehicle trajectory tracking are low, mainly due to the presence of monitoring leakage points at the video splicing, which affects the continuity and accuracy of vehicle trajectory tracking.
By obtaining vehicle traffic videos from multiple different road sections, analyzing vehicle speeds and determining positional adjacent relationships, grouping videos at a unified frame rate, extracting vehicle identity characteristics and driving section order, dividing gap types, and according to gap paths and vehicle driving The state fills the video gap and builds a complete vehicle trajectory.
It improves the adaptability and accuracy of vehicle trajectory tracking, ensures the integrity and reliability of vehicle trajectory, reduces leakage points in video analysis, and improves the efficiency of intelligent traffic management systems.
Smart Images
Figure CN118968779B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video analysis, and in particular to a vehicle trajectory tracking method and system based on video analysis. Background Art
[0002] The vehicle trajectory tracking solution based on video analysis mainly relies on the rapid development of computer vision and artificial intelligence technology. Traditional traffic monitoring relies on a lot of human analysis, which is inefficient and costly. With the improvement of video processing technology, the use of mixed Gaussian models for background modeling, background difference method to detect moving vehicles, and combined with particle filtering and other methods to achieve real-time tracking of vehicle trajectories has become a mainstream trend. This solution not only reduces the burden of manpower, but also realizes the real-time processing of traffic accidents and violations, and improves the efficiency and accuracy of the intelligent traffic management system.
[0003] In the prior art, each video is often a traffic video that can only represent the traffic conditions of a certain section of road. The vehicle trajectory and driving status are tracked by splicing the traffic videos of all sections of the road. However, there are often monitoring omissions at the splicing points of each video, that is, there is no traffic video at that location or on that section of the road, and there are gaps at the video splicing points, resulting in low adaptability and accuracy of vehicle trajectory tracking.
[0004] Therefore, how to improve the adaptability and accuracy of vehicle trajectory tracking is a technical problem that needs to be solved. Summary of the invention
[0005] The purpose of the present invention is to solve the problem of low adaptability and accuracy of vehicle trajectory tracking in the prior art, and to propose a vehicle trajectory tracking method and system based on video analysis, the method comprising:
[0006] Obtain multiple different road section vehicle traffic videos and vehicle drivable paths within the area to be tracked, determine the vehicle speed of each road section by analyzing the vehicle traffic video of each road section, determine the positional proximity relationship between the road sections according to the vehicle drivable paths, and group the different road section vehicle traffic videos according to the vehicle speeds of the adjacent road sections, count the vehicle speeds in each group to set the group frame rate, and convert all road section vehicle traffic videos in the same group into the same frame rate according to the group frame rate, so as to facilitate video feature extraction;
[0007] Extract vehicle identity features from the vehicle traffic video of each road section, and identify and match the vehicles in the vehicle traffic video of the road section based on the vehicle identity features, determine and mark the vehicle appearance frames and vehicle disappearance frames in the vehicle traffic video of each road section, and determine the vehicle driving section sequence based on the positional adjacent relationship between each road section and the vehicle's travel direction;
[0008] The gap types are divided according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of two adjacent road section vehicle traffic videos in the vehicle driving section sequence, and the vehicle traffic videos of each road section are spliced through the vehicle disappearance frame and the vehicle appearance frame of each road section vehicle traffic video and the positional adjacent relationship between each road section to obtain a comprehensive road section vehicle traffic video, and the gap types are marked in the comprehensive road section vehicle traffic video;
[0009] Other vehicle features and road section features in the comprehensive road section vehicle traffic video are extracted, and the vehicle driving status and vehicle trajectory of each road section in the comprehensive road section vehicle traffic video are generated by combining other vehicle features and road section features. The gap path corresponding to the gap is determined according to the vehicle's drivable path, and the gap is filled by the gap path, gap type, and the vehicle driving status and vehicle trajectory of the road sections before and after the gap, so as to construct a complete driving trajectory of the vehicle.
[0010] In some embodiments of the present application, the vehicle speed of each road section is determined by analyzing the vehicle traffic video of each road section, including:
[0011] A matching relationship between the scene position in the road section vehicle traffic video and the actual road section position is established. The position information of the vehicle in the road section vehicle traffic video in continuous frames is detected through a computer vision algorithm, and the pixel speed is calculated according to the position change of the vehicle in continuous frames. The actual vehicle speed is obtained based on the matching relationship between the pixel speed and the scene position in the road section vehicle traffic video and the actual road section position, so as to obtain the average speed of all vehicles in each road section, and the average speed is used as the vehicle speed of the section.
[0012] In some embodiments of the present application, different road section vehicle traffic videos are grouped according to the speed of adjacent road sections, and the speed of each group is counted to set the group frame rate, including:
[0013] Calculate the speed change value of two adjacent road sections, traverse all road sections based on the speed of the road sections, and divide the road sections with adjacent positions and similar speeds into the same group, so as to group the vehicle traffic videos of different road sections. Calculate the average speed in each group according to the speed of each road section and the number of road sections in each group, and determine the group frame rate by mapping the average speed in each group.
[0014] In some embodiments of the present application, the sequence of the vehicle's travel sections is determined by the positional proximity relationship between the sections and the vehicle's travel direction, including:
[0015] Two adjacent road sections are determined based on the positional proximity relationship between the road sections. The vehicle travel direction of the tracked vehicle is determined based on the order of timestamps of the tracked vehicle appearing in the road section vehicle traffic video corresponding to the two adjacent road sections or the changes in timestamps of consecutive frames in the road section vehicle traffic video. All road section vehicle traffic videos where the tracked vehicle has appeared are determined, and the vehicle travel section sequence of the tracked vehicle is determined in combination with the vehicle travel direction.
[0016] In some embodiments of the present application, the gap type is divided according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of two adjacent road sections in the sequence of vehicle driving sections, including:
[0017] Two adjacent sections in the sequence of the vehicle driving sections are divided into a front section and a rear section, and the time difference and actual distance between the vehicle disappearance frame in the section vehicle traffic video of the front section and the vehicle appearance frame in the section vehicle traffic video of the rear section are calculated;
[0018] If the actual distance is less than the distance threshold, the gap between the vehicle disappearance frame and the vehicle appearance frame is a short gap, and the short gap is further divided into a normal short gap and an extremely short gap according to the time difference;
[0019] If the actual distance is not less than the distance threshold, the gap between the vehicle disappearance frame and the vehicle appearance frame is a long gap, and the long gap is further divided into a normal long gap and an extra long gap according to the time difference.
[0020] In some embodiments of the present application, the vehicle traffic videos of each road section are spliced by the vehicle disappearance frames and vehicle appearance frames of each road section vehicle traffic video and the positional adjacent relationship between each road section to obtain a comprehensive road section vehicle traffic video, and the gap type is marked in the comprehensive road section vehicle traffic video, including:
[0021] The vehicle disappearance frame in the road section vehicle traffic video of the front road section and the vehicle appearance frame in the road section vehicle traffic video of the rear road section are spliced, and a gap segment is reserved between the vehicle disappearance frame and the vehicle appearance frame, and the gap type is marked on the gap segment, so as to obtain a comprehensive road section vehicle traffic video with gap segments and gap types.
[0022] In some embodiments of the present application, the gaps are filled by the gap path, gap type, and the vehicle driving status and vehicle trajectory of the road sections before and after the gaps, so as to construct a complete driving trajectory of the vehicle, including:
[0023] Obtaining road information corresponding to the gap path, and obtaining the road section characteristics corresponding to the gap path according to the road information, wherein the vehicle driving state includes driving speed and driving acceleration;
[0024] For normal short gaps, the gap path is linearly interpolated according to the vehicle disappearance frame and vehicle appearance frame of the road section vehicle traffic video of the road section before and after the gap, so as to obtain the vehicle driving state and vehicle trajectory of the gap path;
[0025] For particularly short gaps, the driving speed and acceleration of the vehicle disappearance frame and vehicle appearance frame of the road section vehicle traffic video of the road section before and after the gap and the road section characteristics of the gap path are input into the preset road condition scenario simulation model, and the vehicle driving state and vehicle trajectory of the gap path are output;
[0026] For normal long gaps and particularly long gaps, the gaps are filled according to the road section characteristics of the gap path, the vehicle driving status and vehicle trajectory of the road sections before and after the gap, and the vehicle driving status and vehicle trajectory of the gap path are obtained;
[0027] The vehicle driving status and vehicle trajectory of each road section and the vehicle driving status and vehicle trajectory of the gap path are integrated to construct a complete vehicle driving trajectory.
[0028] In some embodiments of the present application, for normal long gaps and particularly long gaps, the gaps are filled according to the road section characteristics of the gap path, the vehicle driving status and vehicle trajectory of the road sections before and after the gap, and the vehicle driving status and vehicle trajectory of the gap path are obtained, including:
[0029] For a normal long gap, multiple comparison vehicles other than the tracking vehicle that traveled the same gap path are identified from the road section vehicle traffic video of the road section before and after the gap, and the vehicle driving status and vehicle trajectory of the multiple comparison vehicles in the road section before and after the gap are determined. The time duration of the tracking vehicle and the comparison vehicle passing through the gap path is determined respectively through the timestamps of the vehicle disappearance frame and the vehicle appearance frame of the road section before and after the gap, and the congestion degree of the gap path is determined in combination with the road section characteristics of the gap path and the gap path length;
[0030] ;
[0031] in, is the congestion degree of the gap path, To track the elapsed time of the vehicle passing through the gap path, To compare the number of vehicles, For the The time it takes to compare vehicles passing through the gap path, is the length of the gap path, is the elapsed time threshold of the gap path, represents the elapsed time threshold of the gap path obtained by mapping the length of the gap path, is the number of segment features of the gap path, For the The congestion response weight of the road section characteristics, For the The quantified value of the road segment characteristics, is a preset constant;
[0032] Find a comparison vehicle that is closest to the tracking vehicle among the comparison vehicles according to the vehicle driving state and vehicle trajectory of the comparison vehicle on the road sections before and after the gap, the timestamp of the comparison vehicle on the gap path, and the vehicle characteristics of the comparison vehicle, and fill in the vehicle driving state and vehicle trajectory of the tracking vehicle on the gap path according to the closest comparison vehicle, the vehicle driving state and vehicle trajectory of the road sections before and after the gap of the tracking vehicle, and the congestion of the gap path;
[0033] For particularly long gaps, multiple comparison vehicles with driving states close to the tracking vehicle are found among the comparison vehicles based on the vehicle driving states of the comparison vehicles on the road sections before and after the gaps. The vehicle driving states and vehicle trajectories of the tracking vehicle on the gap path are filled in based on the selected comparison vehicles and the vehicle driving states and vehicle trajectories of the road sections before and after the gaps of the tracking vehicle.
[0034] Correspondingly, the present application also provides a vehicle trajectory tracking system based on video analysis, comprising:
[0035] The first module is used to obtain vehicle traffic videos of multiple different road sections and vehicle drivable paths within the area to be tracked, determine the vehicle speed of each road section by analyzing the vehicle traffic video of each road section, determine the positional proximity relationship between the road sections according to the vehicle drivable paths, and group the vehicle traffic videos of different road sections according to the vehicle speeds of the adjacent road sections, count the vehicle speeds in each group to set the group frame rate, and convert all the vehicle traffic videos of the same road section in the same group into the same frame rate according to the group frame rate, so as to facilitate video feature extraction;
[0036] The second module is used to extract vehicle identity features from the vehicle traffic video of each road section, and identify and match the vehicles in the vehicle traffic video of the road section based on the vehicle identity features, determine and mark the vehicle appearance frames and vehicle disappearance frames in the vehicle traffic video of each road section, and determine the vehicle driving section sequence according to the positional adjacent relationship between each road section and the vehicle's travel direction;
[0037] The third module is used to classify the gap type according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of two adjacent road sections in the sequence of vehicle driving sections, splice the vehicle traffic videos of each road section through the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of each road section and the position adjacent relationship between each road section, obtain the comprehensive road section vehicle traffic video, and mark the gap type in the comprehensive road section vehicle traffic video;
[0038] The fourth module is used to extract other vehicle features and road section features in the comprehensive road section vehicle traffic video, combine other vehicle features and road section features to generate the vehicle driving status and vehicle trajectory of each road section in the comprehensive road section vehicle traffic video, determine the gap path corresponding to the gap according to the vehicle's drivable path, fill the gap through the gap path, gap type and the vehicle driving status and vehicle trajectory of the road sections before and after the gap, so as to construct a complete driving trajectory of the vehicle.
[0039] By applying the above technical solutions, the present application groups vehicle traffic videos of different sections by the speed of adjacent sections, and sets the group frame rate by counting the speed of each group to unify the video frame rate within the same group, thereby improving the quality and effect of video analysis and providing a basis for subsequent video analysis. The gap types are divided according to the attributes between the two frames of the vehicle disappearance frame and the vehicle appearance frame, and different types of gaps are filled or predicted in different ways to improve the adaptability of gap filling. The gaps are filled by the gap path, gap type, and the vehicle driving status and vehicle trajectory of the sections before and after the gap, thereby improving the accuracy of vehicle trajectory tracking, ensuring the reliability of vehicle trajectory tracking, and facilitating the analysis of vehicle trajectory and vehicle driving status. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 A schematic diagram of a process flow of a vehicle trajectory tracking method based on video analysis proposed by the present invention;
[0041] Figure 2 A schematic structural diagram of a vehicle trajectory tracking system based on video analysis proposed by the present invention. DETAILED DESCRIPTION
[0042] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0043] Reference Figure 1 , a vehicle trajectory tracking method based on video analysis, comprising the following steps:
[0044] Step S101, obtain vehicle traffic videos of multiple different road sections and vehicle drivable paths within the area to be tracked, determine the vehicle speed of each road section by analyzing the vehicle traffic video of each road section, determine the positional proximity relationship between the various road sections according to the vehicle drivable paths, and group the vehicle traffic videos of different road sections according to the vehicle speeds of the adjacent road sections, count the vehicle speeds in each group to set the group frame rate, and convert all the vehicle traffic videos of the same road sections in the same group into the same frame rate according to the group frame rate, so as to facilitate video feature extraction.
[0045] In this embodiment, the positional proximity relationship between the various road sections is determined according to the vehicle's drivable path, and the positional proximity relationship is the progressive connectivity relationship between the various road sections. The average speed of the vehicles in the road sections with positional proximity is determined, and the road sections with positional proximity are integrated and classified by the average speed, and multiple road sections are divided into the same group (adjacent position and similar speed), which is helpful for subsequent video stitching and trajectory alignment.
[0046] In some embodiments of the present application, the vehicle speed of each road section is determined by analyzing the vehicle traffic video of each road section, including:
[0047] A matching relationship between the scene position in the road section vehicle traffic video and the actual road section position is established. The position information of the vehicle in the road section vehicle traffic video in continuous frames is detected through a computer vision algorithm, and the pixel speed is calculated according to the position change of the vehicle in continuous frames. The actual vehicle speed is obtained based on the matching relationship between the pixel speed and the scene position in the road section vehicle traffic video and the actual road section position, so as to obtain the average speed of all vehicles in each road section, and the average speed is used as the vehicle speed of the section.
[0048] In this embodiment, the computer vision algorithm includes a target detection algorithm (such as YOLO, SSD, etc.), which identifies the appearance of the same vehicle in different videos by comparing the features of the vehicles in different videos (such as color, shape, license plate, etc.).
[0049] In some embodiments of the present application, different road section vehicle traffic videos are grouped according to the speed of adjacent road sections, and the speed of each group is counted to set the group frame rate, including:
[0050] Calculate the speed change value of two adjacent road sections, traverse all road sections based on the speed of the road sections, and divide the road sections with adjacent positions and similar speeds into the same group, so as to group the vehicle traffic videos of different road sections. Calculate the average speed in each group according to the speed of each road section and the number of road sections in each group, and determine the group frame rate by mapping the average speed in each group.
[0051] In this embodiment, the video of the road section with a faster speed may require a higher frame rate to capture the features of each frame in the video. On the contrary, the road section with a slower speed does not require a very high frame rate to accurately capture. The road sections with adjacent positions and similar speeds are divided into the same group, so as to group the vehicle traffic videos of different road sections. For example, road section a1-road section a2-road section a3-road section a4, if the speed of road section a1 is similar to that of road section a2, and the speed of road section a2 is similar to that of road section a3, then road section a1-road section a2-road section a3 are divided into the same group.
[0052] Step S102, extracting vehicle identity features from the vehicle traffic video of each road section, and identifying and matching the vehicles in the vehicle traffic video of the road section based on the vehicle identity features, determining and marking the vehicle appearance frames and vehicle disappearance frames in the vehicle traffic video of each road section, and determining the vehicle travel section sequence through the positional proximity relationship between the sections and the vehicle travel direction.
[0053] In this embodiment, the computer vision algorithm includes a target detection algorithm (such as YOLO, SSD, etc.), which identifies the appearance of the same vehicle in different videos by comparing the features of the vehicles in different videos (such as color, shape, license plate, etc.), and identifies the tracked vehicle among the vehicles.
[0054] In some embodiments of the present application, the sequence of the vehicle's travel sections is determined by the positional proximity relationship between the sections and the vehicle's travel direction, including:
[0055] Two adjacent road sections are determined based on the positional proximity relationship between the road sections. The vehicle travel direction of the tracked vehicle is determined based on the order of timestamps of the tracked vehicle appearing in the road section vehicle traffic video corresponding to the two adjacent road sections or the changes in timestamps of consecutive frames in the road section vehicle traffic video. All road section vehicle traffic videos where the tracked vehicle has appeared are determined, and the vehicle travel section sequence of the tracked vehicle is determined in combination with the vehicle travel direction.
[0056] In this embodiment, the road section with the previous timestamp is the previous road section, and the road section with the later timestamp is the later road section. The tracked vehicle first passes through the previous road section and then the later road section, and the order of the vehicle driving sections of the tracked vehicle is determined in combination with the vehicle's traveling direction.
[0057] Step S103, dividing the gap type according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of two adjacent sections in the sequence of vehicle driving sections, splicing the vehicle traffic videos of each section through the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of each section and the positional adjacent relationship between each section to obtain a comprehensive section vehicle traffic video, and marking the gap type in the comprehensive section vehicle traffic video.
[0058] In this embodiment, the gap types are divided according to the attributes (time difference, actual distance) between the vehicle disappearance frame and the vehicle appearance frame of the adjacent road section. The gap refers to the distance between two frames or between two road sections. There is no traffic video on this distance to help track the vehicle trajectory. The gap types include normal short gaps and special short gaps, normal long gaps and special long gaps.
[0059] In some embodiments of the present application, the gap type is divided according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of two adjacent road sections in the sequence of vehicle driving sections, including:
[0060] Two adjacent sections in the sequence of the vehicle driving sections are divided into a front section and a rear section, and the time difference and actual distance between the vehicle disappearance frame in the section vehicle traffic video of the front section and the vehicle appearance frame in the section vehicle traffic video of the rear section are calculated;
[0061] If the actual distance is less than the distance threshold, the gap between the vehicle disappearance frame and the vehicle appearance frame is a short gap, and the short gap is further divided into a normal short gap and an extremely short gap according to the time difference;
[0062] If the actual distance is not less than the distance threshold, the gap between the vehicle disappearance frame and the vehicle appearance frame is a long gap, and the long gap is further divided into a normal long gap and an extra long gap according to the time difference.
[0063] In this embodiment, short gaps and long gaps are first divided according to the actual distance between two frames. Short gaps indicate that the distance is short, and long gaps indicate that the distance is long. Then, according to the time difference (the time from the end of the previous section to the beginning of the next section, that is, the time through the gap), it is divided into normal short gaps and special short gaps. Normal short gaps indicate that the distance is short and the time is short, which is a normal situation. Special short gaps indicate that the distance is short, but the time is long, which means that some special events (long stay time or parking, traffic accidents, etc.) occurred in the gap path, affecting the time. Normal long gaps indicate that the distance is long and the time is relatively long, which is a normal situation. Special long gaps indicate that the distance is long, but the time is relatively short, which means that speeding or other special events of acceleration may have occurred on the gap path.
[0064] In some embodiments of the present application, the vehicle traffic videos of each road section are spliced by the vehicle disappearance frames and vehicle appearance frames of each road section vehicle traffic video and the positional adjacent relationship between each road section to obtain a comprehensive road section vehicle traffic video, and the gap type is marked in the comprehensive road section vehicle traffic video, including:
[0065] The vehicle disappearance frame in the road section vehicle traffic video of the front road section and the vehicle appearance frame in the road section vehicle traffic video of the rear road section are spliced, and a gap segment is reserved between the vehicle disappearance frame and the vehicle appearance frame, and the gap type is marked on the gap segment, so as to obtain a comprehensive road section vehicle traffic video with gap segments and gap types.
[0066] Step S104, extracting other vehicle features and road section features in the comprehensive road section vehicle traffic video, combining other vehicle features and road section features to generate vehicle driving status and vehicle trajectory of each road section in the comprehensive road section vehicle traffic video, determining the gap path corresponding to the gap according to the vehicle's drivable path, filling the gap with the gap path, gap type, and the vehicle driving status and vehicle trajectory of the road sections before and after the gap, thereby constructing a complete driving trajectory of the vehicle.
[0067] In this embodiment, the road segment features are features of road conditions, such as flatness, curvature, length of traffic lights, etc. In combination with these features, a trajectory generation algorithm (such as a Kalman filter, a particle filter) is used to construct the vehicle state and vehicle trajectory of each road segment.
[0068] In some embodiments of the present application, the gaps are filled by the gap path, gap type, and the vehicle driving status and vehicle trajectory of the road sections before and after the gaps, so as to construct a complete driving trajectory of the vehicle, including:
[0069] Obtaining road information corresponding to the gap path, and obtaining the road section characteristics corresponding to the gap path according to the road information, wherein the vehicle driving state includes driving speed and driving acceleration;
[0070] For normal short gaps, the gap path is linearly interpolated according to the vehicle disappearance frame and vehicle appearance frame of the road section vehicle traffic video of the road section before and after the gap, so as to obtain the vehicle driving state and vehicle trajectory of the gap path;
[0071] For particularly short gaps, the driving speed and acceleration of the vehicle disappearance frame and vehicle appearance frame of the road section vehicle traffic video of the road section before and after the gap and the road section characteristics of the gap path are input into the preset road condition scenario simulation model, and the vehicle driving state and vehicle trajectory of the gap path are output;
[0072] For normal long gaps and particularly long gaps, the gaps are filled according to the road section characteristics of the gap path, the vehicle driving status and vehicle trajectory of the road sections before and after the gap, and the vehicle driving status and vehicle trajectory of the gap path are obtained;
[0073] The vehicle driving status and vehicle trajectory of each road section and the vehicle driving status and vehicle trajectory of the gap path are integrated to construct a complete vehicle driving trajectory.
[0074] In this embodiment, for normal short gaps, because of the short distance and time, linear interpolation is performed based on the driving speed and driving acceleration of the two points of the gap vehicle disappearance frame and vehicle appearance frame to fill the gap path. For special short gaps, because some special events (such as stop) are involved, the gap path is filled according to the road condition scenario simulation model.
[0075] In some embodiments of the present application, for normal long gaps and particularly long gaps, the gaps are filled according to the road section characteristics of the gap path, the vehicle driving status and vehicle trajectory of the road sections before and after the gap, and the vehicle driving status and vehicle trajectory of the gap path are obtained, including:
[0076] For a normal long gap, multiple comparison vehicles other than the tracking vehicle that traveled the same gap path are identified from the road section vehicle traffic video of the road section before and after the gap, and the vehicle driving status and vehicle trajectory of the multiple comparison vehicles in the road section before and after the gap are determined. The time duration of the tracking vehicle and the comparison vehicle passing through the gap path is determined respectively through the timestamps of the vehicle disappearance frame and the vehicle appearance frame of the road section before and after the gap, and the congestion degree of the gap path is determined in combination with the road section characteristics of the gap path and the gap path length;
[0077] ;
[0078] in, is the congestion degree of the gap path, To track the elapsed time of the vehicle passing through the gap path, To compare the number of vehicles, For the The time it takes to compare vehicles passing through the gap path, is the length of the gap path, is the elapsed time threshold of the gap path, represents the elapsed time threshold of the gap path obtained by mapping the length of the gap path, is the number of segment features of the gap path, For the The congestion response weight of the road section characteristics, For the The quantified value of the road segment characteristics, is a preset constant;
[0079] Find a comparison vehicle that is closest to the tracking vehicle among the comparison vehicles according to the vehicle driving state and vehicle trajectory of the comparison vehicle on the road sections before and after the gap, the timestamp of the comparison vehicle on the gap path, and the vehicle characteristics of the comparison vehicle, and fill in the vehicle driving state and vehicle trajectory of the tracking vehicle on the gap path according to the closest comparison vehicle, the vehicle driving state and vehicle trajectory of the road sections before and after the gap of the tracking vehicle, and the congestion of the gap path;
[0080] For particularly long gaps, multiple comparison vehicles with driving states close to the tracking vehicle are found among the comparison vehicles based on the vehicle driving states of the comparison vehicles on the road sections before and after the gaps. The vehicle driving states and vehicle trajectories of the tracking vehicle on the gap path are filled in based on the selected comparison vehicles and the vehicle driving states and vehicle trajectories of the road sections before and after the gaps of the tracking vehicle.
[0081] In this embodiment, for a normal long gap, because the vehicle travels for a long time and a long distance on the gap path, the closest comparison vehicle is found, and the gap path is filled in by combining the driving status, driving trajectory, and congestion degree of the gap path before and after the comparison vehicle and the tracking vehicle (the filling can be achieved through fitting methods, machine learning models, Kalman filtering, etc.). The above-mentioned comparison vehicle is a vehicle that has appeared on the same screen as the tracking vehicle in the previous section, the gap path, and the subsequent section, and the closest comparison vehicle is selected from these vehicles.
[0082] In this embodiment, a comparison vehicle closest to the tracking vehicle is searched among the comparison vehicles according to the vehicle driving status and vehicle trajectory of the comparison vehicle in the road sections before and after the gap, the timestamp of the comparison vehicle in the gap path, and the vehicle features of the comparison vehicle. Here, the search can be performed based on the comprehensive similarity. The vehicle features refer to the vehicle volume, vehicle structure, etc.
[0083] ;
[0084] in, For the The comprehensive similarity between the comparison vehicle and the tracking vehicle, For the The similarity between the combined vehicle driving state and vehicle trajectory of the comparison vehicle and the tracking vehicle on the previous road section, For the The similarity between the time difference between the vehicle appearance frame and the vehicle disappearance frame of the comparison vehicle and the tracking vehicle on the previous road section, For the The similarity between the comparison vehicle and the tracking vehicle on the gap path is combined with the vehicle disappearance frame and the vehicle appearance frame. For the The similarity between the combined vehicle driving state and vehicle trajectory of the comparison vehicle and the tracking vehicle on the following road section, For the The similarity between the time difference between the vehicle appearance frame and the vehicle disappearance frame of the comparison vehicle and the tracking vehicle on the following road section, , , are the similarity weights of the previous segment, the gap path, and the next segment, respectively.
[0085] In this embodiment, for particularly long gaps, because the driving speed of this path is relatively fast, multiple comparison vehicles with driving states close to the tracking vehicle are searched among the comparison vehicles according to the vehicle driving states of the comparison vehicles on the road sections before and after the gaps, and the similarity of vehicle speed and acceleration is emphasized in the search process. The vehicle driving state and vehicle trajectory of the tracking vehicle on the gap path are filled in according to the selected comparison vehicles and the vehicle driving states and vehicle trajectories of the road sections before and after the gaps of the tracking vehicle (filling can be achieved through fitting methods, machine learning models, Kalman filtering, etc.).
[0086] By applying the above technical scheme, vehicle traffic videos of multiple different road sections and vehicle drivable paths within the area to be tracked are obtained, the vehicle speed of each road section is determined by analyzing the vehicle traffic video of each road section, the positional proximity relationship between each road section is determined according to the vehicle drivable path, and the vehicle traffic videos of different road sections are grouped according to the vehicle speeds of the adjacent road sections, the vehicle speeds in each group are counted to set the group frame rate, and all the vehicle traffic videos of the same road section in the same group are converted into the same frame rate according to the group frame rate, so as to facilitate video feature extraction; vehicle identity features are extracted from each road section vehicle traffic video, and vehicles in the road section vehicle traffic video are identified and matched based on the vehicle identity features, and vehicle appearance frames and vehicle disappearance frames are determined and marked in each road section vehicle traffic video, and the vehicle driving direction is determined by the positional proximity relationship between each road section and the vehicle's travel direction. The sequence of road sections; the gap types are divided according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of two adjacent road sections in the sequence of vehicle driving sections, and the vehicle traffic videos of each road section are spliced through the vehicle disappearance frame and the vehicle appearance frame of each road section vehicle traffic video and the positional adjacent relationship between each road section to obtain a comprehensive road section vehicle traffic video, and the gap types are marked in the comprehensive road section vehicle traffic video; other vehicle features and road section features in the comprehensive road section vehicle traffic video are extracted, and the vehicle driving status and vehicle trajectory of each road section in the comprehensive road section vehicle traffic video are generated by combining other vehicle features and road section features, and the gap path corresponding to the gap is determined according to the vehicle drivable path, and the gap is filled by the gap path, gap type, and the vehicle driving status and vehicle trajectory of the road section before and after the gap, so as to construct a complete driving trajectory of the vehicle. The present application groups different road section vehicle traffic videos by the speed of adjacent road sections, and sets the group frame rate by counting the speed in each group to unify the video frame rate in the same group, thereby improving the quality and effect of video analysis and providing a basis for subsequent video analysis. The gap types are divided according to the attributes between the vehicle disappearance frame and the vehicle appearance frame, and different types of gaps are filled or predicted in different ways to improve the adaptability of gap filling. The gaps are filled by the gap path, gap type, and the vehicle driving status and vehicle trajectory of the road section before and after the gap, thereby improving the accuracy of vehicle trajectory tracking, ensuring the reliability of vehicle trajectory tracking, and facilitating the analysis of vehicle trajectory and vehicle driving status.
[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present invention can be implemented by hardware, or by software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present invention.
[0088] Correspondingly, the present application also provides a vehicle trajectory tracking system based on video analysis, such as Figure 2 As shown, including:
[0089] The first module is used to obtain vehicle traffic videos of multiple different road sections and vehicle drivable paths within the area to be tracked, determine the vehicle speed of each road section by analyzing the vehicle traffic video of each road section, determine the positional proximity relationship between the road sections according to the vehicle drivable paths, and group the vehicle traffic videos of different road sections according to the vehicle speeds of the adjacent road sections, count the vehicle speeds in each group to set the group frame rate, and convert all the vehicle traffic videos of the same road section in the same group into the same frame rate according to the group frame rate, so as to facilitate video feature extraction;
[0090] The second module is used to extract vehicle identity features from the vehicle traffic video of each road section, and identify and match the vehicles in the vehicle traffic video of the road section based on the vehicle identity features, determine and mark the vehicle appearance frames and vehicle disappearance frames in the vehicle traffic video of each road section, and determine the vehicle driving section sequence according to the positional adjacent relationship between each road section and the vehicle's travel direction;
[0091] The third module is used to classify the gap type according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of two adjacent road sections in the sequence of vehicle driving sections, splice the vehicle traffic videos of each road section through the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of each road section and the position adjacent relationship between each road section, obtain the comprehensive road section vehicle traffic video, and mark the gap type in the comprehensive road section vehicle traffic video;
[0092] The fourth module is used to extract other vehicle features and road section features in the comprehensive road section vehicle traffic video, combine other vehicle features and road section features to generate the vehicle driving status and vehicle trajectory of each road section in the comprehensive road section vehicle traffic video, determine the gap path corresponding to the gap according to the vehicle's drivable path, fill the gap through the gap path, gap type and the vehicle driving status and vehicle trajectory of the road sections before and after the gap, so as to construct a complete driving trajectory of the vehicle.
[0093] Those skilled in the art will appreciate that the modules in the system in the implementation scenario can be distributed in the system of the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more systems different from the implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple submodules.
[0094] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A vehicle trajectory tracking method based on video analysis, characterized in that: include: Obtain multiple different road section vehicle traffic videos and vehicle drivable paths within the area to be tracked, determine the vehicle speed of each road section by analyzing the vehicle traffic video of each road section, determine the positional proximity relationship between the road sections according to the vehicle drivable paths, and group the different road section vehicle traffic videos according to the vehicle speeds of the adjacent road sections, count the vehicle speeds in each group to set the group frame rate, and convert all road section vehicle traffic videos in the same group into the same frame rate according to the group frame rate, so as to facilitate video feature extraction; Extract vehicle identity features from the vehicle traffic video of each road section, and identify and match the vehicles in the vehicle traffic video of the road section based on the vehicle identity features, determine and mark the vehicle appearance frames and vehicle disappearance frames in the vehicle traffic video of each road section, and determine the vehicle driving section sequence based on the positional adjacent relationship between each road section and the vehicle's travel direction; The gap types are divided according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of two adjacent road section vehicle traffic videos in the vehicle driving section sequence, and the vehicle traffic videos of each road section are spliced through the vehicle disappearance frame and the vehicle appearance frame of each road section vehicle traffic video and the positional adjacent relationship between each road section to obtain a comprehensive road section vehicle traffic video, and the gap types are marked in the comprehensive road section vehicle traffic video; Other vehicle features and road section features in the comprehensive road section vehicle traffic video are extracted, and the vehicle driving status and vehicle trajectory of each road section in the comprehensive road section vehicle traffic video are generated by combining other vehicle features and road section features. The gap path corresponding to the gap is determined according to the vehicle's drivable path, and the gap is filled by the gap path, gap type, and the vehicle driving status and vehicle trajectory of the road sections before and after the gap, so as to construct a complete driving trajectory of the vehicle.
2. The vehicle trajectory tracking method based on video analysis according to claim 1, characterized in that: The speed of each road segment is determined by analyzing the vehicle traffic video of each road segment, including: A matching relationship between the scene position in the road section vehicle traffic video and the actual road section position is established. The position information of the vehicle in the road section vehicle traffic video in continuous frames is detected through a computer vision algorithm, and the pixel speed is calculated according to the position change of the vehicle in continuous frames. The actual vehicle speed is obtained based on the matching relationship between the pixel speed and the scene position in the road section vehicle traffic video and the actual road section position, so as to obtain the average speed of all vehicles in each road section, and the average speed is used as the vehicle speed of the section.
3. The vehicle trajectory tracking method based on video analysis according to claim 2 is characterized in that: The vehicle traffic videos of different sections are grouped according to the speed of the adjacent sections, and the speed of each group is counted to set the group frame rate, including: Calculate the speed change value of two adjacent road sections, traverse all road sections based on the speed of the road sections, and divide the road sections with adjacent positions and similar speeds into the same group, so as to group the vehicle traffic videos of different road sections. Calculate the average speed in each group according to the speed of each road section and the number of road sections in each group, and determine the group frame rate by mapping the average speed in each group.
4. The vehicle trajectory tracking method based on video analysis according to claim 1, characterized in that: The sequence of vehicle travel sections is determined by the positional proximity relationship between each section and the vehicle's travel direction, including: Two adjacent road sections are determined based on the positional proximity relationship between the road sections. The vehicle travel direction of the tracked vehicle is determined based on the order of timestamps of the tracked vehicle appearing in the road section vehicle traffic video corresponding to the two adjacent road sections or the changes in timestamps of consecutive frames in the road section vehicle traffic video. All road section vehicle traffic videos where the tracked vehicle has appeared are determined, and the vehicle travel section sequence of the tracked vehicle is determined in combination with the vehicle travel direction.
5. The vehicle trajectory tracking method based on video analysis according to claim 1, characterized in that: The gap types are classified according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of two adjacent road sections in the sequence of vehicle driving sections, including: Two adjacent sections in the sequence of the vehicle driving sections are divided into a front section and a rear section, and the time difference and actual distance between the vehicle disappearance frame in the section vehicle traffic video of the front section and the vehicle appearance frame in the section vehicle traffic video of the rear section are calculated; If the actual distance is less than the distance threshold, the gap between the vehicle disappearance frame and the vehicle appearance frame is a short gap, and the short gap is further divided into a normal short gap and an extremely short gap according to the time difference; If the actual distance is not less than the distance threshold, the gap between the vehicle disappearance frame and the vehicle appearance frame is a long gap, and the long gap is further divided into a normal long gap and an extra long gap according to the time difference.
6. The vehicle trajectory tracking method based on video analysis according to claim 5 is characterized in that: The vehicle traffic videos of each road section are spliced through the vehicle disappearance frames and vehicle appearance frames of the vehicle traffic video of each road section and the positional adjacency relationship between each road section to obtain a comprehensive road section vehicle traffic video, and the gap types are marked in the comprehensive road section vehicle traffic video, including: The vehicle disappearance frame in the road section vehicle traffic video of the front road section and the vehicle appearance frame in the road section vehicle traffic video of the rear road section are spliced, and a gap segment is reserved between the vehicle disappearance frame and the vehicle appearance frame, and the gap type is marked on the gap segment, so as to obtain a comprehensive road section vehicle traffic video with gap segments and gap types.
7. The vehicle trajectory tracking method based on video analysis according to claim 6 is characterized in that: The gaps are filled by the gap path, gap type, and the vehicle driving status and vehicle trajectory of the road segments before and after the gaps, so as to construct a complete driving trajectory of the vehicle, including: Obtaining road information corresponding to the gap path, and obtaining the road section characteristics corresponding to the gap path according to the road information, wherein the vehicle driving state includes driving speed and driving acceleration; For normal short gaps, the gap path is linearly interpolated according to the vehicle disappearance frame and vehicle appearance frame of the road section vehicle traffic video of the road section before and after the gap, so as to obtain the vehicle driving state and vehicle trajectory of the gap path; For particularly short gaps, the driving speed and acceleration of the vehicle disappearance frame and vehicle appearance frame of the road section vehicle traffic video of the road section before and after the gap and the road section characteristics of the gap path are input into the preset road condition scenario simulation model, and the vehicle driving state and vehicle trajectory of the gap path are output; For normal long gaps and particularly long gaps, the gaps are filled according to the road section characteristics of the gap path, the vehicle driving status and vehicle trajectory of the road sections before and after the gap, and the vehicle driving status and vehicle trajectory of the gap path are obtained; The vehicle driving status and vehicle trajectory of each road section and the vehicle driving status and vehicle trajectory of the gap path are integrated to construct a complete vehicle driving trajectory.
8. The vehicle trajectory tracking method based on video analysis according to claim 7 is characterized in that: For normal long gaps and particularly long gaps, the gaps are filled according to the road section characteristics of the gap path, the vehicle driving status and vehicle trajectory of the road sections before and after the gap, and the vehicle driving status and vehicle trajectory of the gap path are obtained, including: For a normal long gap, multiple comparison vehicles other than the tracking vehicle that traveled the same gap path are identified from the road section vehicle traffic video of the road section before and after the gap, and the vehicle driving status and vehicle trajectory of the multiple comparison vehicles in the road section before and after the gap are determined. The time duration of the tracking vehicle and the comparison vehicle passing through the gap path is determined respectively through the timestamps of the vehicle disappearance frame and the vehicle appearance frame of the road section before and after the gap, and the congestion degree of the gap path is determined in combination with the road section characteristics of the gap path and the gap path length; ； in, is the congestion degree of the gap path, To track the elapsed time of the vehicle passing through the gap path, To compare the number of vehicles, For the The time it takes to compare vehicles passing through the gap path, is the length of the gap path, is the elapsed time threshold of the gap path, represents the elapsed time threshold of the gap path obtained by mapping the length of the gap path, is the number of segment features of the gap path, For the The congestion response weight of the road section characteristics, For the The quantified value of the road segment characteristics, is a preset constant; Find a comparison vehicle that is closest to the tracking vehicle among the comparison vehicles according to the vehicle driving state and vehicle trajectory of the comparison vehicle on the road sections before and after the gap, the timestamp of the comparison vehicle on the gap path, and the vehicle characteristics of the comparison vehicle, and fill in the vehicle driving state and vehicle trajectory of the tracking vehicle on the gap path according to the closest comparison vehicle, the vehicle driving state and vehicle trajectory of the road sections before and after the gap of the tracking vehicle, and the congestion of the gap path; For particularly long gaps, multiple comparison vehicles with driving states close to the tracking vehicle are found among the comparison vehicles based on the vehicle driving states of the comparison vehicles on the road sections before and after the gaps. The vehicle driving states and vehicle trajectories of the tracking vehicle on the gap path are filled in based on the selected comparison vehicles and the vehicle driving states and vehicle trajectories of the road sections before and after the gaps of the tracking vehicle.
9. A vehicle trajectory tracking system based on video analysis, characterized in that: include: The first module is used to obtain vehicle traffic videos of multiple different road sections and vehicle drivable paths within the area to be tracked, determine the vehicle speed of each road section by analyzing the vehicle traffic video of each road section, determine the positional proximity relationship between the road sections according to the vehicle drivable paths, and group the vehicle traffic videos of different road sections according to the vehicle speeds of the adjacent road sections, count the vehicle speeds in each group to set the group frame rate, and convert all the vehicle traffic videos of the same road section in the same group into the same frame rate according to the group frame rate, so as to facilitate video feature extraction; The second module is used to extract vehicle identity features from the vehicle traffic video of each road section, and identify and match the vehicles in the vehicle traffic video of the road section based on the vehicle identity features, determine and mark the vehicle appearance frames and vehicle disappearance frames in the vehicle traffic video of each road section, and determine the vehicle driving section sequence according to the positional adjacent relationship between each road section and the vehicle's travel direction; The third module is used to classify the gap type according to the attributes between the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of two adjacent road sections in the sequence of vehicle driving sections, splice the vehicle traffic videos of each road section through the vehicle disappearance frame and the vehicle appearance frame of the vehicle traffic video of each road section and the position adjacent relationship between each road section, obtain the comprehensive road section vehicle traffic video, and mark the gap type in the comprehensive road section vehicle traffic video; The fourth module is used to extract other vehicle features and road section features in the comprehensive road section vehicle traffic video, combine other vehicle features and road section features to generate the vehicle driving status and vehicle trajectory of each road section in the comprehensive road section vehicle traffic video, determine the gap path corresponding to the gap according to the vehicle's drivable path, fill the gap through the gap path, gap type and the vehicle driving status and vehicle trajectory of the road sections before and after the gap, so as to construct a complete driving trajectory of the vehicle.