All-weather efficient road traffic condition evaluation system with lane self-recognition function
By using the YOLOv8 model and BoTSORT algorithm in the traffic detection system, combining non-maximum suppression and reinforcement learning technology, automatically filtering the detection frame and tracking trajectory, the problems of high deployment costs and reduced performance in the existing technology are solved, and the problem of in-depth development of the existing technology is achieved. Efficient traffic conditions assessment in complex weather.
Patent Information
- Application Number
- CN202411278994.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-11-22
AI Technical Summary
When using deep learning models, the deployment cost of existing traffic detection systems has increased significantly and performance deteriorated in complex weather conditions, and multiple hyperparameters need to be manually adjusted to suit different scenarios.
A full-weather and efficient road traffic condition evaluation system for lane self-identification is designed, using the YOLOv8 model combined with the non-maximum suppression algorithm for vehicle detection, and automatically filters the detection frame and trajectory through preset parameters based on reinforcement learning. The BoTSORT algorithm tracks the vehicle trajectory, combines the trajectory angle distribution to separate lanes, and evaluates traffic conditions.
It realizes adaptive work in complex weather, reduces deployment costs, avoids manual labeling and parameter adjustment, and improves the efficiency and accuracy of the system.
Smart Images

Figure CN119007447A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traffic assessment, and more specifically, to an all-weather efficient road traffic condition assessment system with lane self-identification. Background Art
[0002] There are existing traffic status assessment systems designed for public webcams. For example, a low-budget real-time traffic congestion detection system uses Haar-like features to detect vehicles based on online images. A paper proposed an improved vehicle detection system that uses a neural network-based YOLOv3 model for vehicle detection and estimates the current traffic status by measuring mIOU (mean intersection over union). However, the system only accepts some low-frame rate images from webcams as input.
[0003] With the rapid progress of neural network research, some traffic state detection works designed based on multi-target tracking deep learning methods have emerged. Some papers point out that advanced target tracking algorithms are essential to significantly improve the performance of the entire system. However, some algorithms are not designed for traffic cameras, and often multiple hyperparameters may need to be properly adjusted to adapt to the input scene for optimal performance. However, manually marking and adjusting the hyperparameters of road information for each scene greatly increases the deployment cost. In addition, some target detection algorithms require careful adjustment of parameters to work well in rainy and snowy weather, at dusk or at night.
[0004] At the same time, in recent years, with the continuous upgrading of network cameras, the frame rate and resolution of video streams have continued to increase, and the computing resources required to process these videos have grown rapidly. For example, the old camera with 6 frames per minute has been replaced by a new camera with 30 frames per second, or 1800 frames per minute. The resolution has also been upgraded from 360P (480×360 pixels) to 1080P (1920×1080 pixels). In other words, the number of pixels that need to be processed per minute has increased by 3600 times. To process such data, more and more powerful hardware is required, and the cost has been greatly increased. It can be expected that traffic cameras will be upgraded regularly and the cost of data processing will continue to increase. Summary of the invention
[0005] The present invention provides an all-weather and efficient road traffic condition assessment system with lane self-identification, which solves the technical problem that in the prior art, deep learning models are commonly used in traffic detection systems using cameras. The latest models have better effects, but will significantly increase the system deployment cost.
[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:
[0007] An all-weather efficient road traffic condition assessment system with lane self-identification, comprising:
[0008] A video data acquisition module, the video data acquisition module is used to acquire traffic video data including vehicles;
[0009] A vehicle detection module, the vehicle detection module is used to detect vehicles in the traffic video data;
[0010] A vehicle trajectory tracking module, wherein the vehicle trajectory tracking module is used to track the trajectories of multiple detected vehicles;
[0011] A lane detection module, the lane detection module is used to determine the lane to which the trajectory of each vehicle belongs;
[0012] A traffic condition evaluation module is used to evaluate the traffic condition of a given lane based on the time required to pass through the given lane and the travel time of the vehicle's trajectory.
[0013] Furthermore, the vehicle detection module is used to detect vehicles in the traffic video data, including:
[0014] The vehicle detection module uses the YOLOv8 model to detect vehicles and combines the non-maximum suppression algorithm to obtain the optimal detection frame of the vehicle.
[0015] Furthermore, the method of obtaining the optimal detection frame of the vehicle by combining the non-maximum suppression algorithm includes:
[0016] Run the non-maximum suppression algorithm using loose non-maximum suppression parameters to obtain a preliminary detection frame;
[0017] The preliminary detection frame is filtered using a heuristic filtering strategy to obtain an optimal detection frame for the vehicle.
[0018] Furthermore, the heuristic filtering strategy includes:
[0019] First, filter the detection frames that do not belong to the vehicle category in the preliminary detection frame;
[0020] The second time, among the detection frames remaining after the first filtering, the detection frames whose sizes are smaller than the minimum size threshold and whose sizes are larger than the maximum size threshold are filtered;
[0021] The third time, the detection frames remaining after the second filtering and whose confidence is less than the confidence threshold are filtered to obtain the best detection frame of the vehicle.
[0022] Furthermore, the minimum size threshold, maximum size threshold and confidence threshold are obtained through a preset parameter adjustment model based on reinforcement learning.
[0023] Furthermore, the vehicle trajectory tracking module is used to track the trajectories of the detected multiple vehicles, including:
[0024] Use the BoTSORT algorithm to obtain preliminary trajectories of multiple vehicles;
[0025] According to the confidence level of each trajectory, erroneous trajectories in the preliminary trajectories of the multiple vehicles are deleted to obtain trajectories of multiple vehicles.
[0026] Further, according to the confidence level of each trajectory, deleting the erroneous trajectory in the preliminary trajectories of the plurality of vehicles comprises:
[0027] Define the trajectory as where p i =(i,x,y),p i is a point on the trajectory detected by the video frame with frame number i. The position of the point is (x, y). For a trajectory, i0 is the first detected frame, i n is the last detected frame, then, add the following rule to determine whether the created track has high confidence:
[0028] Filter the trajectories whose length is less than the minimum length threshold and greater than the maximum length threshold;
[0029] Filter the trajectories whose trajectory point density is less than the minimum trajectory point density threshold;
[0030] Filter the trajectories whose point jump amplitude is greater than the maximum point jump amplitude threshold.
[0031] Furthermore, the minimum length threshold, the maximum length threshold, the minimum trajectory point density threshold and the maximum point jump amplitude threshold are obtained through a pre-trained reinforcement learning model.
[0032] Furthermore, the lane detection module is used to determine the lane to which the trajectory of each vehicle belongs, including:
[0033] Represented by vector<p0,pn> Project the trajectory, and then use the unit vector <0,1> to calculate its angle θ. By performing trajectory angle statistics over time, we can get the trajectory angle distribution curve. Assuming that the driving angle of each lane always obeys a distribution, the trajectory angles of N lanes should follow a set of N distributions. The minimum value of the trajectory angle distribution curve is used as the threshold for separating lanes. The lanes are separated and the lane to which each vehicle's trajectory belongs is obtained.
[0034] Furthermore, the traffic condition evaluation module evaluates the traffic condition of a given lane according to the time required to pass through the given lane and the travel time of the trajectory of the vehicle, including:
[0035]
[0036] In the formula, when When it is less than 0, it means the traffic is smooth. When it is greater than or equal to 1, it indicates traffic congestion; t represents the median travel time of all vehicles in the preset time period, t min The median of the shortest 10% travel times on the previous day, t max =αt min , where α can be used to adjust the congestion judgment threshold.
[0037] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0038] The system structure of the present invention is similar to that of a standard traffic detection system, including data processing, vehicle detection and tracking, and traffic status estimation. Through a large number of improvements, the system of the present invention can efficiently and adaptively work in complex weather conditions, and there is no need to perform any road marking or parameter settings for the camera during deployment. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 A schematic diagram of the modules of a lane self-identification, all-weather, efficient road traffic condition assessment system provided by an embodiment of the present invention.
[0040] Figure 2 A schematic diagram of a process for obtaining an optimal detection frame for a vehicle provided by an embodiment of the present invention;
[0041] Figure 3 A schematic diagram of a process for tracking the trajectories of multiple vehicles detected according to an embodiment of the present invention. DETAILED DESCRIPTION
[0042] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;
[0043] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;
[0044] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0045] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0046] Example
[0047] An all-weather and efficient road traffic condition assessment system with lane self-identification, such as Figure 1 As shown, including:
[0048] A video data acquisition module, the video data acquisition module is used to acquire traffic video data including vehicles;
[0049] A vehicle detection module, the vehicle detection module is used to detect vehicles in the traffic video data;
[0050] A vehicle trajectory tracking module, wherein the vehicle trajectory tracking module is used to track the trajectories of multiple detected vehicles;
[0051] A lane detection module, the lane detection module is used to determine the lane to which the trajectory of each vehicle belongs;
[0052] A traffic condition evaluation module is used to evaluate the traffic condition of a given lane based on the time required to pass through the given lane and the travel time of the vehicle's trajectory.
[0053] In a specific embodiment, this embodiment can efficiently and adaptively work in complex weather conditions through a large number of improvements, and does not require any road marking or parameter settings for the camera during deployment, including:
[0054] Unique system structure
[0055] The video source is directly sent to the corresponding GPU memory in the cluster without caching for use
[0056] Parallel real-time object detection using in-memory data sources
[0057] Dynamically select target detection models of different complexity based on server load to balance load and improve efficiency
[0058] Improved NMS strategy and more information provided by target detectors in complex weather conditions
[0059] · Filtering algorithm including untrustworthy and duplicate vehicle detection boxes
[0060] Non-lane area monitoring method based on statistical information
[0061] ·Parameter selection strategy for detection box filtering algorithm
[0062] Parallel and asynchronous vehicle appearance and motion feature extraction
[0063] Automatically select target tracking algorithms of different complexity based on the length of the data queue to balance the load and improve efficiency
[0064] Trajectory filtering algorithm designed for traffic state estimation
[0065] Parameter selection strategy for trajectory filtering algorithm
[0066] Automatic lane discovery algorithm based on statistics
[0067] Lane number estimation method in lane discovery algorithm
[0068] Design of noise filter for statistical data in lane finding algorithm
[0069] Lane change detection method in lane discovery algorithm
[0070] · Calculation method of vehicle passing time in camera field of view
[0071] Traffic state estimation method based on passing time
[0072] In a further embodiment, the vehicle detection module is used to detect vehicles in the traffic video data, including:
[0073] The vehicle detection module uses the YOLOv8 model to detect vehicles and combines the non-maximum suppression algorithm to obtain the optimal detection frame of the vehicle.
[0074] In a specific example, there are several viable options for detecting vehicles in a video. Popular object detection models are able to achieve high accuracy on specific datasets. However, some models are slower and may require multiple GPUs to perform vehicle detection in camera videos. If a larger model can process 20 frames per second, and the input source generates 200 frames per second, this will result in a 10x cost difference.
[0075] YOLO is a popular choice in object detection. Different versions have been released by different research teams over the past few years. Therefore, this example compares the performance differences between multiple different models. Among these different versions of YOLO, YOLOv8 performed best in the test using the same video frame.
[0076] Ultralytics YOLOv8 improves upon the previous successful YOLO versions, introducing new features and performance improvements to quickly and accurately perform object detection tasks. YOLOv8 is implemented in the PyTorch framework, and its pre-trained models are easy to use and well suited for different engineering applications. Several model sizes are available in YOLOv8, with YOLOv8n reaching 1000FPS, YOLOv8m reaching 546FPS, and YOLOv8x being able to process 283FPS on an A100 device. Therefore, there are multiple options in the YOLOv8 model to initially meet the requirements for performance and fast processing speeds.
[0077] In a further embodiment, the non-maximum suppression algorithm is combined to obtain the best detection frame of the vehicle, such as Figure 2 As shown, including:
[0078] Run the non-maximum suppression algorithm using loose non-maximum suppression parameters to obtain a preliminary detection frame;
[0079] The preliminary detection frame is filtered using a heuristic filtering strategy to obtain an optimal detection frame for the vehicle.
[0080] In a specific embodiment, the number of objects detected by the object detection algorithm may exceed the actual number. The non-maximum suppression (NMS) algorithm is widely used as a post-processing step for object detection. In a given scenario, by properly adjusting the parameters, the algorithm can find the best detection box among all possible choices. Street cameras are in a dynamic environment, such as street lighting and weather changes. This makes the optimal NMS parameters in a state of constant change.
[0081] If the NMS parameters are set more strictly, vehicles at dusk or on rainy days are more likely to be missed. If these parameters are more relaxed, there may be too many duplicate detection frames on sunny days. If the NMS parameters need to be manually adjusted for a large number of cameras one by one to adapt to different situations, real-time adjustments will be too costly and unacceptable.
[0082] There have been some studies focusing on this problem and several algorithms have been proposed. For example, adaptive NMS or using a neural network model to estimate the density score for dynamic adjustment of the threshold. This works well in high-density tracking. The disadvantage of this approach is that it requires training a new model, which increases the computational power requirements for real-time detection, thereby increasing the cost.
[0083] Since the system of this embodiment focuses on the overall traffic status rather than a single vehicle, this enables this embodiment to propose an optimized NMS strategy in the system. This embodiment proposes a two-stage NMS strategy. First, NMS is run with loose parameters, which may result in more detection boxes than the actual number of objects, and then this embodiment uses a heuristic filtering strategy for execution.
[0084] Since this embodiment focuses only on vehicles, this embodiment excludes detection frames that have been identified as pedestrians, animals, or ships. All street cameras are aimed at a section of road and are not close to any specific vehicle. Therefore, the size of the vehicle detection frame should be within a reasonable range and usually does not occupy the entire screen or only a few pixels. This embodiment can exclude detection frames that are too large or too small. At the same time, this embodiment also excludes vehicles that are too far away, because vehicles that are farther away need to travel a longer distance to move one pixel on the camera, and tracking vehicles that are farther away will increase system errors.
[0085] In a further embodiment, the heuristic filtering strategy includes:
[0086] First, filter the detection frames that do not belong to the vehicle category in the preliminary detection frame;
[0087] The second time, among the detection frames remaining after the first filtering, the detection frames whose sizes are smaller than the minimum size threshold and whose sizes are larger than the maximum size threshold are filtered;
[0088] The third time, the detection frames remaining after the second filtering and whose confidence is less than the confidence threshold are filtered to obtain the best detection frame of the vehicle.
[0089] In a further embodiment, the minimum size threshold, the maximum size threshold and the confidence threshold are obtained by a preset parameter adjustment model based on reinforcement learning.
[0090] In a specific embodiment, the selection of the threshold is an important factor affecting the effect of the algorithm. This embodiment trains a small parameter adjustment model based on reinforcement learning, specifically:
[0091] Based on the detection box data, a reinforcement learning model is constructed, whose input is the detection box information (position, size), and the output is the parameters, including the minimum size threshold, maximum size threshold and confidence threshold. These parameters are used to filter the detection box and manually estimate the score of the filtered detection box. This score is used as the value function for model update. Through repeated training, after reaching a stable state, a model that estimates the necessary parameters based on the detection box information is obtained. Its reinforcement learning model only includes a policy network, which is manually estimated as the value function. The state of its environment is the input trajectory data. Its action is the output parameter set. Its reward is a manually estimated mechanism function. The training process is as follows:
[0092] 1. Initialization: Initialize the network and randomly select a batch of trajectories as the initial state
[0093] 2. Interactive learning: For each batch of trajectories, input them into the model and get parameter output. Use the output parameters to filter these trajectories. Manually evaluate the quality of the trajectories and give rewards. Use the rewards to update the network.
[0094] 3. Loop: Repeat the interactive learning steps until the model performance is reliable and stable.
[0095] Looser NMS parameters may reduce missed detections in bad weather, but a small number of missed detections will not affect the overall trajectory. However, in the subsequent tracking process, continued missed detections cannot be fixed because there is no early trajectory information. At the same time, this may result in multiple detection boxes and multiple trajectories on a road. However, in the subsequent traffic status detection, this embodiment uses statistical data saved in a short time window to calculate the traffic status. Multiple trajectories of a vehicle will slightly affect the calculation results, but will hardly affect the judgment of whether the traffic is congested.
[0096] In a further embodiment, the vehicle trajectory tracking module is used to track the trajectories of multiple detected vehicles, such as Figure 3 As shown, including:
[0097] Use the BoTSORT algorithm to obtain preliminary trajectories of multiple vehicles;
[0098] According to the confidence level of each trajectory, erroneous trajectories in the preliminary trajectories of the multiple vehicles are deleted to obtain trajectories of multiple vehicles.
[0099] In a specific embodiment, tracking of multiple vehicle trajectories is a common multi-target tracking problem. SORT (Simple Online and Realtime Tracking) based algorithms are the most popular algorithm category. SORT-based improved algorithms, such as BoTSORT (Boosted Tracklets with SORT) and DeepSORT (Deep Association Network with SORT), show good results and speed in real-time target tracking under different appearance and motion models.
[0100] However, the situation becomes complicated when considering the influence of specific environmental conditions. For example, strong sea winds often cause camera shake in the analyzed online videos. This instability causes nonlinear motion of the vehicle in the video frames and changes the aspect ratio of the vehicle bounding box due to the change in viewpoint. These factors often lead to inaccurate trajectory results.
[0101] BoTSORT is now used in the new tracker of this embodiment. This embodiment selected BoTSORT from multiple trackers taking into account computational workload and performance. As mentioned earlier, BoTSORT is also similar to SORT. It optimizes the state vector of the Kalman filter based on DeepSORT and improves performance by directly estimating the width and height of the bounding box without using any aspect ratio. To address the jitter problem of street cameras, global motion compensation is introduced, and sparse registration technology is used to ignore dynamic objects in the scene to more accurately estimate background motion. Finally, this embodiment uses an optimized strategy to fuse appearance information with motion data, so that the system of this embodiment can produce accurate trajectory derivations.
[0102] These improvements have enabled BoTSORT to achieve state-of-the-art performance in the MOT17 and MOT20 challenges, and it remains one of the highest performing trackers to date. Compared to algorithms that use larger models and introduce more neural network models, BoTSORT adds very little computational effort but achieves significant performance gains.
[0103] In addition, this embodiment uses a Kalman filter-based motion model and a pre-trained appearance neural network from the MOT16 challenge. All real-time vehicle tracking trajectories are saved. In addition to estimating traffic status, the dynamically established trajectories are also used to estimate and create lane information at low cost. Experiments have shown that the design proposed in this embodiment is fast, but accuracy is sacrificed, which may result in some erroneous trajectory generation.
[0104] This example compares BoTSORT and DeepSORT for vehicle trajectory detection. In the comparison, this example uses the smallest and lowest performing model, YOLOLv8n, as a detector on a 10-minute video. This is because this example hopes to find better detection performance with the simplest tracking strategy. The resulting trajectory BoTSORT performs better. Each line drawn in a random color represents a trajectory. BoTSORT is able to produce higher quality trajectories when the detection quality is limited.
[0105] Specifically, on this tracking segment, the trajectory generated by DeepSORT is discontinuous due to camera shaking. DeepSORT obtained 69 trajectories, but the average length was 10.36. On the other hand, BoTSORT obtained 34 higher quality trajectories with an average length of 68.17 under the same conditions. Although BoTSORT requires more computing operations, in the system of this embodiment, the bottleneck of speed is the deep neural network model. Therefore, BoTSORT does not reduce the overall inference speed and can obtain more high-quality trajectories.
[0106] In a further embodiment, deleting erroneous trajectories in the preliminary trajectories of the plurality of vehicles according to the confidence level of each trajectory comprises:
[0107] Define the trajectory as where p i =(i,x,y),p i is a point on the trajectory detected by the video frame with frame number i. The position of the point is (x, y). For a trajectory, i0 is the first detected frame, i n is the last detected frame, then, add the following rule to determine whether the created track has high confidence:
[0108] Filter the trajectories whose length is less than the minimum length threshold and greater than the maximum length threshold;
[0109] Filter the trajectories whose trajectory point density is less than the minimum trajectory point density threshold;
[0110] Filter the trajectories whose point jump amplitude is greater than the maximum point jump amplitude threshold.
[0111] In a specific embodiment, since MOT (multi-object tracking) is still a fundamental problem in computer vision, some erroneous trajectories are inevitable. However, most of the errors in tracking come from the performance limitations of the model selected in the detection phase. This is because the present embodiment does not select the most accurate model, but rather a model that balances accuracy and speed. When the number of reported vehicles increases, the present embodiment may obtain obviously incorrect trajectories. Some trajectories obtained from real-time video are not suitable for traffic state estimation.
[0112] In the traffic state estimation problem, it is completely different from the MOT problem. For example, the IDs of the front and rear vehicles in a common trajectory are swapped. This is an unacceptable error in the standard MOT design, but it hardly affects the average elapsed time. Therefore, this embodiment uses a different design concept from MOT when designing the trajectory filtering strategy.
[0113] In a further embodiment, the minimum length threshold, the maximum length threshold, the minimum trajectory point density threshold and the maximum point jump amplitude threshold are obtained through a pre-trained reinforcement learning model.
[0114] In a specific embodiment, a reinforcement learning model is constructed based on trajectory data, whose input is trajectory information and output is the parameters, including the minimum length threshold, maximum length threshold, minimum trajectory point density threshold and maximum point jump amplitude threshold. These parameters are used to filter the trajectory and manually estimate the quality score of the filtered trajectory. This score is used as the value function for model updating. By repeated training, after reaching a stable state, a model is obtained that estimates the necessary parameters based on trajectory information, and its reinforcement learning model only includes a policy network, which is manually estimated as a value function. The state of its environment is the input trajectory data. Its action is the output parameter set. Its reward is a manually estimated mechanism function. The training process is as follows:
[0115] 1. Initialization: Initialize the network and randomly select a batch of trajectories as the initial state
[0116] 2. Interactive learning: For each batch of trajectories, input them into the model and get parameter output. Use the output parameters to filter these trajectories. Manually evaluate the quality of the trajectories and give rewards. Use the rewards to update the network.
[0117] 3. Loop: Repeat the interactive learning steps until the model performance is reliable and stable.
[0118] In a further embodiment, the lane detection module is used to determine the lane to which the trajectory of each vehicle belongs, including:
[0119] Represented by vector<p0,pn> Project the trajectory, and then use the unit vector <0,1> to calculate its angle θ. By performing trajectory angle statistics over time, we can get the trajectory angle distribution curve. Assuming that the driving angle of each lane always obeys a distribution, the trajectory angles of N lanes should follow a set of N distributions. The minimum value of the trajectory angle distribution curve is used as the threshold for separating lanes. The lanes are separated and the lane to which each vehicle's trajectory belongs is obtained.
[0120] In a specific embodiment, after obtaining the trajectory of the vehicle in the video, directly estimating the overall congestion state may still be inaccurate because the traffic status of different directions and lanes may be different. This embodiment should first determine the lane to which each trajectory belongs and calculate the traffic status of each lane. Some systems recommend using manually marked lanes, but if manual marking is applied to a system with a large number of video streams, it will be cumbersome and expensive. In addition, some lanes may be subject to traffic control by the police or used for different driving directions at different times of the day.
[0121] In order to automatically identify and distinguish lane trajectories, traffic signs on the road can be identified, but this is actually a difficult method. In addition, the method may not be universal in different regions or countries. Therefore, the present embodiment proposes to use a tracking angle-based method to distinguish lanes. The method operates from the perspective of the camera, and even lanes that are parallel to each other are larger in the closer part, smaller in the farther part, and converge at the vanishing point. If the starting point and end point of each lane trajectory are connected, they should have different visual angles. Therefore, parallel lanes appear at different angles on the screen. The present embodiment can use trajectory angles to identify different lanes.
[0122] This is an automatic process, so any changes in the road are automatically detected. Mathematically, it can be represented by the vector<p_0,p_n> Project the trajectory and then use the unit vector <0,1> to calculate its angle θ. By performing trajectory angle statistics over time, its distribution can be obtained. Assuming that the driving angle of each lane always follows a distribution, the trajectory angles of N lanes should follow a set of N distributions.
[0123] Based on this assumption, an angle-based lane separation mechanism is feasible. The minimum value of the trajectory angle distribution curve is used as the threshold for lane separation. However, in practical applications, due to the limited number of lanes, the distribution function may be noisy, so the statistical data should be filtered before using the minimum value as the lane separation threshold.
[0124] Some exceptions. For vehicles that change lanes, the angle-based algorithm prefers to identify the trajectory as the final lane, since the lane change causes the angle of the vehicle trajectory to increase. For vehicle trajectories that are stopped in the middle of the road or cannot be tracked, they are removed in the previous step of the trajectory screening strategy, as they should not be included in the subsequent passing time calculation.
[0125] In a further embodiment, the traffic status of the lane can be observed by measuring the travel time of the trajectory. The travel time is defined as the number of frames that have passed, and the time required to pass through a given lane in the video is analyzed and determined using historical data. This embodiment uses the lane tracking results of the previous day to calculate the vehicle's travel time, and takes the median of the shortest 10% travel time as the smoothed travel time, recorded as t min To define congested or slow-moving traffic conditions, define t max =αt min , where α can be used to adjust the congestion judgment threshold;
[0126] The traffic condition evaluation module evaluates the traffic condition of a given lane according to the time required to pass through the given lane and the travel time of the trajectory of the vehicle, including:
[0127]
[0128] In the formula, when When it is less than 0, it means the traffic is smooth. When it is greater than or equal to 1, it indicates traffic congestion; t represents the median travel time of all vehicles in the preset time period, t min The median of the shortest 10% travel times on the previous day, t max =αt min , where α can be used to adjust the congestion judgment threshold.
[0129] The same or similar reference numerals correspond to the same or similar components;
[0130] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;
[0131] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. An all-weather efficient road traffic condition assessment system with lane self-identification, characterized in that: include: A video data acquisition module, the video data acquisition module is used to acquire traffic video data including vehicles; A vehicle detection module, the vehicle detection module is used to detect vehicles in the traffic video data; A vehicle trajectory tracking module, wherein the vehicle trajectory tracking module is used to track the trajectories of multiple detected vehicles; A lane detection module, the lane detection module is used to determine the lane to which the trajectory of each vehicle belongs; A traffic condition evaluation module is used to evaluate the traffic condition of a given lane based on the time required to pass through the given lane and the travel time of the vehicle's trajectory.
2. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 1 is characterized in that: The vehicle detection module is used to detect vehicles in the traffic video data, including: The vehicle detection module uses the YOLOv8 model to detect vehicles and combines the non-maximum suppression algorithm to obtain the optimal detection frame of the vehicle.
3. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 2 is characterized in that: The method of combining the non-maximum suppression algorithm to obtain the optimal detection frame of the vehicle includes: Run the non-maximum suppression algorithm using loose non-maximum suppression parameters to obtain a preliminary detection frame; The preliminary detection frame is filtered using a heuristic filtering strategy to obtain an optimal detection frame for the vehicle.
4. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 3 is characterized in that: The heuristic filtering strategy includes: First, filter the detection frames that do not belong to the vehicle category in the preliminary detection frame; The second time, among the detection frames remaining after the first filtering, the detection frames whose sizes are smaller than the minimum size threshold and whose sizes are larger than the maximum size threshold are filtered; The third time, the detection frames remaining after the second filtering and whose confidence is less than the confidence threshold are filtered to obtain the best detection frame of the vehicle.
5. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 4 is characterized in that: The minimum size threshold, the maximum size threshold and the confidence threshold are obtained through a preset parameter adjustment model based on reinforcement learning.
6. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 1 is characterized in that: The vehicle trajectory tracking module is used to track the trajectories of multiple detected vehicles, including: Use the BoTSORT algorithm to obtain preliminary trajectories of multiple vehicles; According to the confidence level of each trajectory, erroneous trajectories in the preliminary trajectories of the multiple vehicles are deleted to obtain trajectories of multiple vehicles.
7. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 6 is characterized in that: Deleting an erroneous track from the preliminary tracks of the plurality of vehicles according to a confidence level of each track, comprising: Define the trajectory as where p i =(i,x,y),p i is a point on the trajectory detected by the video frame with frame number i. The position of the point is (x, y). For a trajectory, i0 is the first detected frame, i n is the last detected frame, then, add the following rule to determine whether the created track has high confidence: Filter the trajectories whose length is less than the minimum length threshold and greater than the maximum length threshold; Filter the trajectories whose trajectory point density is less than the minimum trajectory point density threshold; Filter the trajectories whose point jump amplitude is greater than the maximum point jump amplitude threshold.
8. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 7 is characterized in that: The minimum length threshold, the maximum length threshold, the minimum trajectory point density threshold and the maximum point jump amplitude threshold are obtained through a pre-trained reinforcement learning model.
9. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 1, characterized in that: The lane detection module is used to determine the lane to which the trajectory of each vehicle belongs, including: Represented by vector<p0,pn> Project the trajectory, and then use the unit vector <0,1> to calculate its angle θ. By performing trajectory angle statistics over time, we can get the trajectory angle distribution curve. Assuming that the driving angle of each lane always obeys a distribution, the trajectory angles of N lanes should follow a set of N distributions. The minimum value of the trajectory angle distribution curve is used as the threshold for separating lanes. The lanes are separated and the lane to which each vehicle's trajectory belongs is obtained.
10. The all-weather efficient road traffic condition assessment system with lane self-identification according to claim 9, characterized in that: The traffic condition evaluation module evaluates the traffic condition of a given lane according to the time required to pass through the given lane and the travel time of the trajectory of the vehicle, including: In the formula, when When it is less than 0, it means the traffic is smooth. When it is greater than or equal to 1, it indicates traffic congestion; t represents the median travel time of all vehicles in the preset time period, t min The median of the shortest 10% travel times on the previous day, t max =αt min , where α can be used to adjust the congestion judgment threshold.
Citation Information
Cited By
Traffic flow statistics and congestion analysis method and system based on video analysis
CN119889057A
Method, system and equipment for detecting road vehicle speed and vehicle distance based on unmanned aerial vehicle vision
CN120877528A
