Dynamic traffic target detection tracking and forward collision early warning method

Abstract

The invention discloses a dynamic traffic target detection tracking and forward collision early warning method, and solves the problems of low target detection precision, poor tracking stability and high real-time requirement in a complex traffic environment. By improving the YOLOv8 network, the small target detection precision and the lightweight performance of the network are improved; and the tracking process of DeepSORT is optimized by using a complete intersection-parallel ratio, so that the tracking effect of the vehicle in a rapid moving and shielding scene is improved. In addition, the distance between the vehicles and the collision time are calculated in real time based on the Mobileye model, and three-section type risk early warning is achieved. The vehicle collision early warning method has the advantages of high detection precision, strong real-time performance, wide application and the like, and is suitable for vehicle collision early warning tasks in automatic driving and intelligent traffic systems.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving and intelligent transportation, and in particular to a method for dynamic traffic target detection and tracking and forward collision warning. Background Art

[0002] With the development of autonomous driving technology and the popularization of intelligent transportation systems, vehicle target detection and real-time tracking technology have become key components to ensure driving safety. The existing YOLO series of target detection algorithms are widely used in vehicle detection due to their real-time and high efficiency. DeepSORT, as a classic multi-target tracking algorithm, can perform real-time vehicle tracking based on the target's spatial position, motion trajectory and other characteristics.

[0003] However, in actual application scenarios, especially in complex traffic environments such as highways and congested urban roads, the existing YOLO and DeepSORT algorithms have some shortcomings. For example, insufficient target detection accuracy: When detecting small targets at a long distance and obscured vehicles, the traditional YOLO network has deficiencies in feature extraction and target positioning, resulting in reduced detection accuracy. Tracking stability issues: In multi-target tracking scenarios, DeepSORT is limited by the traditional intersection-over-union matching algorithm, and is prone to target loss or mismatching, especially when the target moves quickly or the vehicle size changes. Real-time requirements: Existing algorithms find it difficult to balance detection accuracy and processing speed in complex traffic scenarios, and cannot meet the needs of efficient real-time processing, affecting the actual application effect.

[0004] Therefore, it is an urgent need in the current technical field to propose a dynamic traffic target detection, tracking and collision warning method that can maintain efficient detection and stable tracking in complex environments and has real-time warning functions. Summary of the invention

[0005] The purpose of the present invention is to solve the problems of complex background, fast vehicle movement speed, difficulty in detecting small targets and high real-time requirements in vehicle target detection and tracking in dynamic traffic scenes, and propose a dynamic traffic target detection and tracking and forward collision warning method, which can effectively improve the detection accuracy of small targets at a distance and fast-moving targets in complex traffic environments, and avoid false detection and missed detection problems. At the same time, combined with lightweight model design and parameter-free attention mechanism, the real-time processing capability and detection efficiency are greatly improved, so that the method shows robust performance in dynamic traffic scenes, meeting the requirements of autonomous driving and intelligent transportation systems for high precision and low latency.

[0006] To achieve the above purpose, the technical solution provided by the present invention is: a dynamic traffic target detection and tracking and forward collision warning method, which realizes accurate detection and real-time tracking of the target vehicle in front by the driving vehicle through the improved YOLOv8 network and the improved DeepSORT algorithm; wherein, the improvement of the YOLOv8 network is to enhance the feature fusion module, the attention module and the detection loss function module, in the feature fusion module, the feature maps of different scales extracted by the backbone network of the YOLOv8 network are lightweight fused, and a parameter-free attention mechanism is introduced in the attention module to dynamically adjust the weights of each channel, emphasize important features and suppress irrelevant and redundant features, and introduce a weighted intersection-over-union ratio as a detection loss function in the detection loss function module; the improvement of the DeepSORT algorithm is to enhance the tracking loss function module, and use the complete intersection-over-union ratio as the tracking loss function to optimize the distance between the tracking bounding box and the real bounding box to reduce the mismatch phenomenon;

[0007] The specific implementation of the method includes the following steps:

[0008] 1) Obtaining video image data from the driving recorder of the vehicle being driven;

[0009] 2) Use the trained improved YOLOv8 network and improved DeepSORT algorithm to process the video image data as follows:

[0010] Input the video image data into the backbone network of the improved YOLOv8 network to obtain feature map A; input feature map A into the feature fusion module to obtain the multi-scale feature map B; input the multi-scale feature map B into the attention module to obtain the weighted feature map C; input the weighted feature map C into the detection head of the improved YOLOv8 network to obtain the target vehicle bounding box;

[0011] The target vehicle bounding box is input into the tracking feature extraction module of the improved DeepSORT algorithm, and the identity feature vector corresponding to the target vehicle bounding box is output; the target vehicle bounding box and the identity feature vector are input into the motion prediction module of the improved DeepSORT algorithm to obtain the predicted bounding box position E; the predicted bounding box position E is input into the matching module of the improved DeepSORT algorithm to obtain the continuous tracking result of the target vehicle, that is, the vehicle continuous tracking bounding box;

[0012] 3) Based on the pixel size of the vehicle's continuous tracking bounding box and the physical size of the actual vehicle, the Mobileye model calculates the real-time distance between the target vehicle and the vehicle being driven. The collision time is then calculated based on the real-time distance and the relative speed of the target vehicle and the vehicle being driven. Finally, based on the relationship between the collision time and the pre-set safety time, the risk level is divided into three levels of risk: high, medium, and no risk, and real-time warning information is output.

[0013] Furthermore, in step 1), the video image data of the road ahead of the vehicle is collected in real time through the driving recorder of the vehicle being driven, and the video image data is preprocessed, including denoising, image resizing and frame rate standardization, to meet the computing requirements of the improved YOLOv8 network and the improved DeepSORT algorithm.

[0014] Further, the improved YOLOv8 network includes:

[0015] A feature fusion module, which includes a lightweight convolutional layer for extracting multi-scale features and upsampling and downsampling units for scale conversion;

[0016] The attention module consists of a non-parametric computation unit and a non-parametric attention mechanism, including convolutional layers and sigmoid activation layers;

[0017] Detection loss function module, which includes weighted intersection-over-union loss function for measuring the weights of various targets; backbone network, which consists of 5 convolutions; layer group and residual structure group

[0018] The detection head consists of 3 convolutional layers, sigmoid activation function and batch normalization.

[0019] Furthermore, the feature fusion module specifically performs the following operations:

[0020] Lightweight convolution operation:

[0021] F conv =DepthwiseConv(A) (1)

[0022] Upsampling and downsampling unit operations:

[0023] F up =Up(F Conv ), F down = Down(F Conv ) (2)

[0024] Multi-scale feature fusion:

[0025] B=F up +F Conv +F Down (3)

[0026] In the formula, DepthwiseConv represents a lightweight convolution operation, Up and Down represent upsampling and downsampling operations respectively; F up 、F Conv 、F Down They represent the multi-scale feature map B obtained after upsampling, lightweight convolution operation and downsampling respectively;

[0027] The attention module takes the multi-scale feature map B as input and obtains the weighted feature map C through calculation. The calculation process is as follows:

[0028] C=B·σ(Conv 1×1 (Concat(AvgPool(B),MaxPool(B)))) (4)

[0029] Where AvgPool(B) represents the average pooling of the multi-scale feature map B along the channel dimension; MaxPool(B) represents the maximum pooling of the multi-scale feature map B along the channel dimension; Concat(·) represents feature concatenation; Conv 1×1 (·) represents a 1×1 convolution on the concatenated feature map, and σ(·) represents the Sigmoid activation function;

[0030] The calculation process of the weighted intersection-over-union loss function TotalLoss of the detection loss function module is as follows:

[0031]

[0032]

[0033] Where N represents the total number of targets, w class_i represents the weight factor of the i-th target category class_i, X pi , X ti Represent the predicted bounding box and the true bounding box of the i-th target respectively; |X pi ∩X ti | represents the intersection area between the predicted bounding box and the true bounding box of the i-th target; |X pi ∪X ti | represents the union area of ​​the predicted bounding box and the true bounding box of the i-th target, L IOU_i represents the intersection-over-union loss of the i-th target.

[0034] Further, the improved DeepSORT algorithm includes:

[0035] A tracking feature extraction module, which consists of lightweight convolutional layers to extract identity features from the target vehicle bounding box;

[0036] The motion prediction module consists of a Kalman filter that estimates the motion state of the target and calculates the predicted bounding box position E;

[0037] A matching module, which includes a Hungarian matching algorithm and a cost matrix unit, associates the detection results of the current frame with the trajectory of the previous frame;

[0038] The tracking loss function module consists of a full intersection-over-union loss function to ensure the consistency of the target position in consecutive frames.

[0039] Furthermore, the target vehicle bounding box is input into the tracking feature extraction module to obtain the identity feature vector d corresponding to each bounding box. i , the specific steps are as follows:

[0040] d i =Conv(R i )(7)

[0041] In the formula, represents the vector space, represents the identity feature vector of target i, D is the dimension of the feature vector, R i represents the vehicle bounding box of target i;

[0042] The target vehicle bounding box is input into the motion prediction module, which uses the Kalman filter to estimate the predicted bounding box position E of the target in the next frame. The expression formula is as follows:

[0043]

[0044] In the formula, F represents the state transfer matrix, X t-1 represents the target vehicle bounding box at the last moment, K represents the Kalman gain, Z t represents the detection bounding box at the current moment, H represents the observation matrix, P t represents the updated covariance matrix, I represents the identity matrix, P t-1 represents the covariance matrix of the previous moment, F T represents the transpose of the state transfer matrix, Q represents the process noise covariance matrix;

[0045] The predicted bounding box position E and the identity feature vector of the detected bounding box are input into the matching module. The matching module calculates the cost matrix between the two and selects the detected bounding box with the smallest cost matrix as the tracking bounding box. The expression formula is as follows:

[0046] cost = βcost IOU +βcost dis (9)

[0047]

[0048] In the formula, cost is the total cost, cost IOU is the intersection and union cost, cost dis is the distance cost, b t and Respectively represent the detection bounding box at the current moment and the prediction bounding box at the previous moment, d t-1 and dt Represent the identity feature vectors of the detection bounding box at the previous moment and the target vehicle bounding box at the current moment respectively;

[0049] The tracking bounding box obtained by the improved DeepSORT algorithm is input into the tracking loss function module, which uses the complete intersection-over-union loss function L CIoU The error is calculated by calculating the overlapping area between the tracking bounding box and the real bounding box, and introducing the center point position deviation and the consistency of the frame aspect ratio for a more comprehensive match; among them, the complete intersection-over-union loss function L CIoU The specific calculation formula is:

[0050]

[0051] Where N represents the total number of targets, ρ 2 (b,b gt ) is the Euclidean distance between the center point of the tracking bounding box and the real bounding box, b represents the coordinates of the center point of the tracking bounding box, and b gt represents the coordinates of the center point of the real bounding box; c is the diagonal length of the minimum circumscribed matrix that can accommodate both the tracking bounding box and the real bounding box; α is the weight function; v is the aspect ratio consistency indicator, which is calculated as follows:

[0052]

[0053] In the formula, ω gt 、h gt are the width and height of the true bounding box, ω and h are the width and height of the tracked bounding box, respectively.

[0054] Furthermore, in step 3), the real-time distance Dis and collision time t between the target vehicle and the driving vehicle are calculated by using the Mobileye model, combining the pixel size of the vehicle continuous tracking bounding box and the physical size of the actual vehicle. pre , the specific calculation is as follows:

[0055] The real-time distance Dis is calculated by the focal length F of the camera in the driving recorder, the actual height W of the vehicle, and the pixel size P occupied by the vehicle in the image. The formula is:

[0056]

[0057] Based on the real-time distance Dis between the vehicles and the relative speed V between the driving vehicle and the target vehicle rel , calculate the collision time t pre , where collision time refers to the time required for the driving vehicle to collide with the target vehicle in front under the condition of maintaining the current speed and distance. The calculation formula is:

[0058]

[0059] The kinematic formula is used to calculate the pre-made safety distance Dis_safe required to avoid collision with the vehicle behind during emergency braking when the vehicle is accelerating. The calculation formula is as follows:

[0060]

[0061] Where, T W V is the time delay budget for a vehicle to issue a collision warning in a general risk scenario. host is the speed of the vehicle being driven; T hW is the time delay budget required for the vehicle to issue an emergency collision warning in a high-risk scenario; Tpr is the pressure rise delay, that is, the time required for the brake system to build pressure during emergency braking; A W is the acceleration during emergency braking, and Safe is the safe stopping distance;

[0062] Using the relative velocity V rel And the prefabrication safety distance Dis_safe, the prefabrication safety time t can be calculated as follows:

[0063]

[0064] A three-stage risk assessment method is used to compare the collision time t pre The relationship between the size of the vehicle and the prefabricated safety time t is used to assess the rear-end collision risk and make the following early warning judgments:

[0065]

[0066] When the collision time t pre If the collision time is less than half of the pre-set safety time t, it is judged as high risk and the target vehicle is marked with a red frame for early warning. pre Between the prefabrication safety time t and the prefabrication safety time If the collision time is between t and t, it is judged as low risk and the target vehicle is marked with a yellow frame for early warning. pre If it is greater than the pre-set safety time t, it is judged as risk-free and the target vehicle is marked with a green frame.

[0067] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0068] 1. This invention combines the most advanced target detection technology, neural network optimization technology and real-time tracking technology, significantly improving the target detection and tracking performance in complex dynamic traffic environments. By improving the feature fusion module of the YOLOv8 network and introducing a parameter-free attention mechanism, the detection capability of long-distance and small target vehicles is enhanced. The attention mechanism can dynamically adjust the weights of each feature channel to effectively capture the key features of the vehicle, especially improving the detection accuracy in complex traffic scenarios.

[0069] 2. The present invention optimizes the matching accuracy between the predicted bounding box and the real bounding box in the YOLOv8 network training by introducing weighted intersection-over-union, which greatly reduces false detection and missed detection, especially in scenes with fast motion and large changes in vehicle size. In addition, the introduction of full intersection-over-union in the DeepSORT algorithm improves the stability of multi-target vehicle tracking and can effectively handle complex scenes such as vehicle occlusion and speed changes.

[0070] 3. While optimizing the detection accuracy, the present invention also carries out a lightweight network design to meet the real-time requirements. It adopts lightweight convolution operations and multi-scale feature map fusion technology to greatly improve the computational efficiency of the YOLOv8 network. On the basis of ensuring the detection and tracking accuracy, it realizes efficient real-time processing and can meet the high performance requirements in complex traffic environments.

[0071] 4. This invention successfully applies the improved YOLOv8 network and DeepSORT algorithm to dynamic traffic target detection and real-time tracking scenarios, which can effectively improve the accuracy of vehicle collision warning in traffic safety systems. The Mobileye model calculates the real-time vehicle spacing and collision time, and combines the three-stage risk assessment method to provide an efficient and accurate warning system, which is suitable for intelligent transportation systems, autonomous driving and other fields, has broad application prospects and is worth promoting. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1This is the structural diagram of the improved YOLOv8 network; in the figure, Backbone represents the backbone network module, Neck represents the feature fusion module, Head represents the detection head, CBS represents the basic convolution module, s represents the convolution step, k represents the convolution kernel size, and n represents the number of convolution repetitions; C2f represents multiple convolution layers; SPPF represents the fast spatial pyramid pooling layer; SimAM represents the parameter-free attention mechanism; VoVGSCSP represents a feature fusion structure; Concat represents a splicing operation; Upsample represents an upsampling operation; GSConv represents grouped spatial convolution; 1x1 Conv-4*reg_max represents a 1×1 convolution layer for predicting bounding box regression information, and 1x1 Conv-num_class represents a 1×1 convolution layer for predicting target category information.

[0073] Figure 2 Schematic diagram of the improved DeepSORT algorithm.

[0074] Figure 3 Schematic diagram of the physical model used by the Mobileye model to calculate vehicle spacing.

[0075] Figure 4 The flowchart of the method of the present invention is shown in FIG. DETAILED DESCRIPTION

[0076] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0077] like Figures 1 to 4 As shown, this embodiment discloses a method for dynamic traffic target detection and tracking and forward collision warning, which realizes accurate detection and real-time tracking of the target vehicle in front by the driving vehicle through the improved YOLOv8 network and the improved DeepSORT algorithm; wherein, the improvement of the YOLOv8 network is to enhance the feature fusion module, the attention module and the detection loss function module, in the feature fusion module, the feature maps of different scales extracted by the backbone network of the YOLOv8 network are lightweight fused, and a parameter-free attention mechanism is introduced in the attention module to dynamically adjust the weights of each channel, emphasize important features and suppress irrelevant and redundant features, and introduce a weighted intersection-over-union ratio as a detection loss function in the detection loss function module; the improvement of the DeepSORT algorithm is to enhance the tracking loss function module, and use the complete intersection-over-union ratio as the tracking loss function to optimize the distance between the tracking bounding box and the real bounding box to reduce the mismatch phenomenon;

[0078] The specific implementation of the method includes the following steps:

[0079] 1) Obtaining video image data from the driving recorder of the vehicle being driven;

[0080] The driving recorder of the driving vehicle is used to collect video image data of the road in front of the vehicle in real time, and the video image data is preprocessed, including denoising, image size adjustment and frame rate standardization, to meet the computing requirements of the improved YOLOv8 network and the improved DeepSORT algorithm.

[0081] 2) Use the trained improved YOLOv8 network and improved DeepSORT algorithm to process the video image data as follows:

[0082] Input the video image data into the backbone network of the improved YOLOv8 network to obtain feature map A; input feature map A into the feature fusion module to obtain the multi-scale feature map B; input the multi-scale feature map B into the attention module to obtain the weighted feature map C; input the weighted feature map C into the detection head of the improved YOLOv8 network to obtain the target vehicle bounding box;

[0083] The target vehicle bounding box is input into the tracking feature extraction module of the improved DeepSORT algorithm, and the identity feature vector corresponding to the target vehicle bounding box is output; the target vehicle bounding box and the identity feature vector are input into the motion prediction module of the improved DeepSORT algorithm to obtain the predicted bounding box position E; the predicted bounding box position E is input into the matching module of the improved DeepSORT algorithm to obtain the continuous tracking result of the target vehicle;

[0084] 3) Based on the pixel size of the vehicle's continuous tracking bounding box and the physical size of the actual vehicle, the Mobileye model calculates the real-time distance between the target vehicle and the vehicle being driven. The collision time is then calculated based on the real-time distance and the relative speed of the target vehicle and the vehicle being driven. Finally, based on the relationship between the collision time and the pre-set safety time, the risk level is divided into three levels of risk: high, medium, and no risk, and real-time warning information is output.

[0085] Specifically, the improved YOLOv8 network includes:

[0086] A feature fusion module, which includes a lightweight convolutional layer for extracting multi-scale features and upsampling and downsampling units for scale conversion;

[0087] The attention module consists of non-parametric computational units and non-parametric attention mechanisms, including convolutional layers and sigmoid activation layers;

[0088] Detection loss function module, which includes a weighted intersection-over-union loss function for measuring the weights of various targets;

[0089] The backbone network consists of 5 convolutional layer groups and a residual structure;

[0090] The detection head consists of 3 convolutional layers, sigmoid activation function and batch normalization.

[0091] The feature fusion module specifically performs the following operations:

[0092] Lightweight convolution operation:

[0093] F conv =DepthwiseConv(A) (1)

[0094] Upsampling and downsampling unit operations:

[0095] F up =Up(F Conv ), F down = Down(F Conv ) (2)

[0096] Multi-scale feature fusion:

[0097] B=F up +F Conv +F Down (3)

[0098] In the formula, DepthwiseConv represents a lightweight convolution operation, Up and Down represent upsampling and downsampling operations respectively; F up 、F Conv 、F Down They represent the multi-scale feature map B obtained after upsampling, lightweight convolution operation and downsampling respectively;

[0099] The attention module takes the multi-scale feature map B as input and obtains the weighted feature map C through calculation. The calculation process is as follows:

[0100] C=B·σ(Conv 1×1 (Concat(AvgPool(B),MaxPool(B)))) (4)

[0101] Where AvgPool(B) represents the average pooling of the multi-scale feature map B along the channel dimension; MaxPool(B) represents the maximum pooling of the multi-scale feature map B along the channel dimension; Concat(·) represents feature concatenation; Conv 1×1 (·) represents a 1×1 convolution on the concatenated feature map, and σ(·) represents the Sigmoid activation function;

[0102] The calculation process of the weighted intersection-over-union loss function TotalLoss of the detection loss function module is as follows:

[0103]

[0104] Where N represents the total number of targets, w class_i represents the weight factor of the i-th target category class_i, X pi , X ti Represent the predicted bounding box and the true bounding box of the i-th target respectively; |X pi ∩X ti | represents the intersection area between the predicted bounding box and the true bounding box of the i-th target; |X pi ∪X ti | represents the union area of ​​the predicted bounding box and the true bounding box of the i-th target, L IOU_i represents the intersection-over-union loss of the i-th target.

[0105] The improved DeepSORT algorithm includes:

[0106] A tracking feature extraction module, which consists of lightweight convolutional layers to extract identity features from the target vehicle bounding box;

[0107] The motion prediction module consists of a Kalman filter that estimates the motion state of the target and calculates the predicted bounding box position E;

[0108] A matching module, which includes a Hungarian matching algorithm and a cost matrix unit, associates the detection results of the current frame with the trajectory of the previous frame;

[0109] The tracking loss function module, which consists of the complete intersection-over-union (CIoU) loss function, ensures the consistency of the target position in consecutive frames.

[0110] The target vehicle bounding box is input into the tracking feature extraction module to obtain the identity feature vector d corresponding to each bounding box. i , the specific steps are as follows:

[0111] d i =Conv(R i )(7)

[0112] In the formula, represents the vector space, represents the identity feature vector of target i, D is the dimension of the feature vector, R i represents the vehicle bounding box of target i;

[0113] The target vehicle bounding box is input into the motion prediction module, which uses the Kalman filter to estimate the predicted bounding box position E of the target in the next frame. The specific steps are as follows:

[0114] The target vehicle bounding box is input into the motion prediction module, which uses the Kalman filter to estimate the predicted bounding box position E of the target in the next frame. The expression formula is as follows:

[0115]

[0116] In the formula, F represents the state transfer matrix, X t-1 represents the target vehicle bounding box at the last moment, K represents the Kalman gain, Z t represents the detection bounding box at the current moment, H represents the observation matrix, P t represents the updated covariance matrix, I represents the identity matrix, P t-1 represents the covariance matrix of the previous moment, F T The transpose of the state transfer matrix, Q represents the process noise covariance matrix.

[0117] The predicted bounding box position E and the identity feature vector of the detected bounding box are input into the matching module. The matching module calculates the cost matrix between the two and selects the detected bounding box with the smallest cost matrix as the tracking bounding box. The expression formula is as follows:

[0118] cost = βcost IOU +βcost dis (9)

[0119]

[0120] In the formula, cost is the total cost, cost IOU is the intersection and union cost, cost dis is the distance cost, b t and Respectively represent the detection bounding box at the current moment and the prediction bounding box at the previous moment, d t-1 and d t Represent the identity feature vectors of the detection bounding box at the previous moment and the target vehicle bounding box at the current moment respectively;

[0121] The tracking bounding box obtained by the improved DeepSORT algorithm is input into the tracking loss function module, which uses the complete intersection-over-union loss function L CIoU The error is calculated by calculating the overlapping area between the tracking bounding box and the real bounding box, and introducing the center point position deviation and the consistency of the frame aspect ratio for a more comprehensive match; among them, the complete intersection-over-union loss function L CIoU The specific calculation formula is:

[0122]

[0123] Where N represents the total number of targets, ρ 2 (b,b gt ) is the Euclidean distance between the center point of the tracking bounding box and the real bounding box, b represents the coordinates of the center point of the tracking bounding box, and b gtrepresents the coordinates of the center point of the real bounding box; c is the diagonal length of the minimum circumscribed matrix that can accommodate both the tracking bounding box and the real bounding box; α is the weight function; v is the aspect ratio consistency indicator, which is calculated as follows:

[0124]

[0125] In the formula, ω gt 、h gt are the width and height of the true bounding box, ω and h are the width and height of the tracked bounding box, respectively.

[0126] Combining the pixel size of the vehicle continuous tracking bounding box and the physical size of the actual vehicle, the Mobileye model is used to calculate the real-time distance Dis and collision time t between the target vehicle and the driving vehicle. pre In the attached Figure 3 In the figure, the relative position relationship between the driving vehicle Car1 and the target vehicles Car2 and Car3 in front is shown to illustrate the distance calculation method between the vehicles and the determination of the collision time. Figure 3 middle:

[0127] I: represents the optical center position of the camera, which is installed in front of the driving vehicle Car1 and is used to capture the image of the vehicle in front.

[0128] Dis1 and Dis2: represent the actual distances between the driving vehicle and the target vehicles Car2 and Car3 in front, respectively.

[0129] F: represents the focal length of the camera in the dash cam, which is a known camera parameter used to calculate the relationship between the pixel size on the image and the actual distance.

[0130] H: Indicates the actual height of the vehicle being driven.

[0131] W: Indicates the actual height of the target vehicle.

[0132] P1 and P2: represent the vertical coordinates of different target vehicle imaging planes on the image plane.

[0133] Specifically, the distances Dis1 and Dis2 between vehicles can be calculated by the focal length F of the camera, the actual height W of the vehicle, and the pixel sizes P1 and P2 occupied by the vehicle in the image, as follows:

[0134]

[0135] Take the driving vehicle Car1 and the target vehicle Car2 in front as an example, calculate the relative speed V between the two rel , calculate the collision time t pre, where collision time refers to the time required for the driving vehicle to collide with the target vehicle in front under the condition of maintaining the current speed and distance. The calculation formula is:

[0136]

[0137] The kinematic formula is used to calculate the pre-made safety distance Dis_safe required to avoid collision with the vehicle behind during emergency braking when the vehicle is accelerating. The calculation formula is as follows:

[0138]

[0139] Where, T W V is the time delay budget for a vehicle to issue a collision warning in a general risk scenario. host is the speed of the vehicle being driven; T hW is the time delay budget required for the vehicle to issue an emergency collision warning in a high-risk scenario; Tpr is the pressure rise delay, that is, the time required for the brake system to build pressure during emergency braking; A W is the acceleration during emergency braking, and Safe is the safe stopping distance;

[0140] Using the relative velocity V rel And the prefabrication safety distance Dis_safe, the prefabrication safety time t can be calculated as follows:

[0141]

[0142] A three-stage risk assessment method is used to compare the collision time t pre The relationship between the size of the vehicle and the prefabricated safety time t is used to assess the rear-end collision risk and make the following early warning judgments:

[0143]

[0144] When the collision time t pre If the collision time is less than half of the pre-set safety time t, it is judged as high risk and the target vehicle is marked with a red frame for early warning. pre Between the prefabrication safety time t and the prefabrication safety time If the collision time is between t and t, it is judged as low risk and the target vehicle is marked with a yellow frame for early warning. pre If it is greater than the pre-set safety time t, it is judged as risk-free and the target vehicle is marked with a green frame.

[0145] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A method for dynamic traffic target detection, tracking and forward collision warning, characterized in that: The method realizes accurate detection and real-time tracking of the target vehicle in front by the driving vehicle by improving the YOLOv8 network and the DeepSORT algorithm; the improvement of the YOLOv8 network is to enhance the feature fusion module, the attention module and the detection loss function module, and the feature fusion module performs lightweight fusion on the feature maps of different scales extracted by the backbone network of the YOLOv8 network, and introduces a parameter-free attention mechanism in the attention module to dynamically adjust the weights of each channel, emphasize important features and suppress irrelevant and redundant features, and introduces weighted intersection-over-union as the detection loss function in the detection loss function module; the improvement of the DeepSORT algorithm is to enhance the tracking loss function module, and use the complete intersection-over-union as the tracking loss function to optimize the distance between the tracking bounding box and the real bounding box to reduce the mismatch phenomenon; The specific implementation of the method includes the following steps: 1) Obtaining video image data from the driving recorder of the vehicle being driven; 2) Use the trained improved YOLOv8 network and improved DeepSORT algorithm to process the video image data as follows: Input the video image data into the backbone network of the improved YOLOv8 network to obtain feature map A; input feature map A into the feature fusion module to obtain the multi-scale feature map B; input the multi-scale feature map B into the attention module to obtain the weighted feature map C; input the weighted feature map C into the detection head of the improved YOLOv8 network to obtain the target vehicle bounding box; The target vehicle bounding box is input into the tracking feature extraction module of the improved DeepSORT algorithm, and the identity feature vector corresponding to the target vehicle bounding box is output; the target vehicle bounding box and the identity feature vector are input into the motion prediction module of the improved DeepSORT algorithm to obtain the predicted bounding box position E; the predicted bounding box position E is input into the matching module of the improved DeepSORT algorithm to obtain the continuous tracking result of the target vehicle, that is, the vehicle continuous tracking bounding box; 3) Based on the pixel size of the vehicle's continuous tracking bounding box and the physical size of the actual vehicle, the Mobileye model calculates the real-time distance between the target vehicle and the vehicle being driven. The collision time is then calculated based on the real-time distance and the relative speed of the target vehicle and the vehicle being driven. Finally, based on the relationship between the collision time and the pre-set safety time, the risk level is divided into three levels of risk: high, medium, and no risk, and real-time warning information is output.

2. A method for detecting and tracking dynamic traffic targets and for warning forward collisions according to claim 1, characterized in that: In step 1), the driving recorder of the driving vehicle collects video image data of the road ahead of the vehicle in real time, and preprocesses the video image data, including denoising, image resizing, and frame rate standardization, to meet the computational requirements of the improved YOLOv8 network and the improved DeepSORT algorithm.

3. A method for detecting and tracking dynamic traffic targets and for warning forward collisions according to claim 1, characterized in that: The improved YOLOv8 network includes: A feature fusion module, which includes a lightweight convolutional layer for extracting multi-scale features and upsampling and downsampling units for scale conversion; The attention module consists of a non-parametric computation unit and a non-parametric attention mechanism, including convolutional layers and sigmoid activation layers; Detection loss function module, which includes a weighted intersection-over-union loss function for measuring the weights of various targets; The backbone network consists of 5 convolutional layers and residual structure groups. The detection head consists of 3 convolutional layers, sigmoid activation function and batch normalization.

4. A method for detecting and tracking dynamic traffic targets and for warning forward collisions according to claim 3, characterized in that: The feature fusion module specifically performs the following operations: Lightweight convolution operation: F conv =DepthwiseConv(A)(1) Upsampling and downsampling unit operations: F up =Up(F Conv ),F down =Down(F Conv ) (2) Multi-scale feature fusion: B=F up +F Conv +F Down (3) In the formula, DepthwiseConv represents a lightweight convolution operation, Up and Down represent upsampling and downsampling operations respectively; F up 、F Conv 、F Down They represent the multi-scale feature map B obtained after upsampling, lightweight convolution operation and downsampling respectively; The attention module takes the multi-scale feature map B as input and obtains the weighted feature map C through calculation. The calculation process is as follows: C=B·σ(Conv 1×1 (Concat(AvgPool(B),MaxPool(B)))) (4) Where AvgPool(B) represents the average pooling of the multi-scale feature map B along the channel dimension; MaxPool(B) represents the maximum pooling of the multi-scale feature map B along the channel dimension; Concat(·) represents feature concatenation; Conv 1×1 (·) represents a 1×1 convolution on the concatenated feature map, and σ(·) represents the Sigmoid activation function; The calculation process of the weighted intersection-over-union loss function TotalLoss of the detection loss function module is as follows: Where N represents the total number of targets, w class_i represents the weight factor of the i-th target category class_i, X pi , X ti Represent the predicted bounding box and the true bounding box of the i-th target respectively; |X pi ∩X ti | represents the intersection area between the predicted bounding box and the true bounding box of the i-th target; |X pi ∪X ti | represents the union area of ​​the predicted bounding box and the true bounding box of the i-th target, L IOU_i represents the intersection-over-union loss of the i-th target.

5. A method for detecting and tracking dynamic traffic targets and for warning forward collisions according to claim 1, characterized in that: The improved DeepSORT algorithm includes: A tracking feature extraction module, which consists of lightweight convolutional layers to extract identity features from the target vehicle bounding box; The motion prediction module consists of a Kalman filter that estimates the motion state of the target and calculates the predicted bounding box position E; A matching module, which includes a Hungarian matching algorithm and a cost matrix unit, associates the detection results of the current frame with the trajectory of the previous frame; The tracking loss function module consists of a full intersection-over-union loss function to ensure the consistency of the target position in consecutive frames.

6. A method for detecting and tracking dynamic traffic targets and for warning forward collisions according to claim 5, characterized in that: The target vehicle bounding box is input into the tracking feature extraction module to obtain the identity feature vector d corresponding to each bounding box. i , the specific steps are as follows: d i =Conv(R i )(7) In the formula, represents the vector space, represents the identity feature vector of target i, D is the dimension of the feature vector, R i represents the vehicle bounding box of target i; The target vehicle bounding box is input into the motion prediction module, which uses the Kalman filter to estimate the predicted bounding box position E of the target in the next frame. The expression formula is as follows: In the formula, F represents the state transfer matrix, X t-1 represents the target vehicle bounding box at the last moment, K represents the Kalman gain, Z t represents the detection bounding box at the current moment, H represents the observation matrix, P t represents the updated covariance matrix, I represents the identity matrix, P t-1 represents the covariance matrix of the previous moment, F T represents the transpose of the state transfer matrix, Q represents the process noise covariance matrix; The predicted bounding box position E and the identity feature vector of the detected bounding box are input into the matching module. The matching module calculates the cost matrix between the two and selects the detected bounding box with the smallest cost matrix as the tracking bounding box. The expression formula is as follows: cost=βcost IOU +βcost dis (9) In the formula, cost is the total cost, cost IOU is the intersection and union cost, cost dis is the distance cost, b t and Represent the current moment detection bounding box and the previous moment prediction bounding box, d t-1 and d t Represent the identity feature vectors of the detection bounding box at the previous moment and the target vehicle bounding box at the current moment respectively; The tracking bounding box obtained by the improved DeepSORT algorithm is input into the tracking loss function module, which uses the complete intersection-over-union loss function L CIoU The error is calculated by calculating the overlapping area between the tracking bounding box and the real bounding box, and introducing the center point position deviation and the consistency of the frame aspect ratio for a more comprehensive match; among them, the complete intersection-over-union loss function L CIoU The specific calculation formula is: Where N represents the total number of targets, ρ 2 (b,b gt ) is the Euclidean distance between the center point of the tracking bounding box and the real bounding box, b represents the coordinates of the center point of the tracking bounding box, and b gt represents the coordinates of the center point of the real bounding box; c is the diagonal length of the minimum circumscribed matrix that can accommodate both the tracking bounding box and the real bounding box; α is the weight function; v is the aspect ratio consistency indicator, which is calculated as follows: In the formula, ω gt 、h gt are the width and height of the true bounding box, ω and h are the width and height of the tracked bounding box, respectively.

7. A method for detecting, tracking and warning a dynamic traffic target as well as a forward collision according to claim 1, characterized in that: In step 3), the real-time distance Dis and collision time t between the target vehicle and the driving vehicle are calculated using the Mobileye model, combining the pixel size of the vehicle continuous tracking bounding box and the physical size of the actual vehicle. pre , the specific calculation is as follows: The real-time distance Dis is calculated by the focal length F of the camera in the driving recorder, the actual height W of the vehicle, and the pixel size P occupied by the vehicle in the image. The formula is: Based on the real-time distance Dis between the vehicles and the relative speed V between the driving vehicle and the target vehicle rel , calculate the collision time t pre , where collision time refers to the time required for the driving vehicle to collide with the target vehicle in front under the condition of maintaining the current speed and distance. The calculation formula is: The kinematic formula is used to calculate the pre-made safety distance Dis_safe required to avoid collision with the vehicle behind during emergency braking when the vehicle is accelerating. The calculation formula is as follows: Where, T W V is the time delay budget for a vehicle to issue a collision warning in a general risk scenario. host is the speed of the vehicle being driven; T hW is the time delay budget required for the vehicle to issue an emergency collision warning in a high-risk scenario; Tpr is the pressure rise delay, that is, the time required for the brake system to build pressure during emergency braking; A W is the acceleration during emergency braking, and Safe is the safe stopping distance; Using the relative velocity V rel And the prefabrication safety distance Dis_safe, the prefabrication safety time t can be calculated as follows: A three-stage risk assessment method is used to compare the collision time t pre The relationship between the size of the vehicle and the prefabricated safety time t is used to evaluate the rear-end collision risk and make the following early warning judgments: When the collision time t pre If it is less than half of the pre-set safety time t, it is judged as high risk and the target vehicle is marked with a red frame for early warning; When the collision time t pre Between the prefabrication safety time t and the prefabrication safety time If the risk is between 0 and 1, it is judged as low risk and the target vehicle is marked with a yellow frame for early warning; When the collision time t pre If it is greater than the pre-set safety time t, it is judged as risk-free and the target vehicle is marked with a green frame.

Images