Front vehicle collision prediction method based on vehicle detection and dynamic distance comparison
By using lightweight convolutional neural networks and image fusion technology, combined with target detection and dynamic distance algorithms, the accuracy problem of forward collision warning under severe weather conditions is solved, achieving efficient and accurate forward collision prediction.
Patent Information
- Application Number
- CN202511052166.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-29
- Publication Date
- 2025-11-18
AI Technical Summary
Existing forward collision warning methods lack sufficient detection accuracy under adverse weather conditions, making it difficult to effectively predict the risk of forward collisions.
A weather multi-label classification model based on the lightweight convolutional neural network MobileNet is adopted, combined with visible light and infrared image fusion technology, and target detection is performed using an improved Yolov11 and HTB network structure. Valid vehicles are screened through ROI region reduction and IOU matching algorithms, and forward collision warning is performed by combining dynamic distance storage algorithm.
It improves the accuracy of forward vehicle detection and collision warning under adverse weather conditions, reduces the detection scope of irrelevant vehicles, and achieves efficient forward vehicle collision prediction with limited mobile device resources.
Smart Images
Figure CN120976877A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of vehicle safety early warning technology, and in particular relates to a method for predicting forward collisions based on vehicle detection and dynamic distance comparison. Background Technology
[0002] In the ever-evolving automotive technology landscape, forward collision warning has become a crucial tool for ensuring driving safety. It is a key active safety technology designed to alert drivers promptly when there is a potential collision risk between the vehicle and the vehicle ahead, using methods such as sound, vision, or touch, thereby enabling measures to prevent or mitigate the severe consequences of a rear-end collision. Deep learning technology can automatically identify and classify vehicles, achieving high-precision vehicle detection. Combined with monocular camera ranging technology, this provides a more accurate and comprehensive forward collision prediction solution.
[0003] Current methods typically rely on sensors such as radar, cameras, lidar, and ultrasonic sensors for real-time monitoring to detect the nearest obstacle vehicle and its relative distance within the vehicle's trajectory. Once a potential collision risk is detected, an immediate warning is issued. Based on accurate identification of valid targets, the system analyzes and makes decisions by combining the vehicle's current driving status with the movement of the valid targets, and appropriately reminds the driver to take necessary evasive measures to avoid traffic accidents, thereby improving the vehicle's active safety performance. Therefore, we propose a forward collision prediction method based on vehicle detection and dynamic distance comparison. Summary of the Invention
[0004] The purpose of this invention is to address the aforementioned technical problems by providing a forward collision prediction method based on vehicle detection and dynamic distance comparison, thereby resolving the issues raised in the background section.
[0005] In view of this, the present invention provides a forward collision prediction method based on vehicle detection and dynamic distance comparison, comprising the following steps:
[0006] Step S1: First, train a weather classification model to identify the current weather conditions in real time. Then, adopt different image denoising methods to improve image quality for different weather conditions. After image preprocessing, adopt an improved target detection algorithm to detect road environment targets such as pedestrians and vehicles in front. Finally, give a response based on the target detection results using the dynamic distance between the vehicle and the vehicle in front.
[0007] Step S2: Build a weather multi-label classification model using MobileNet, a lightweight convolutional neural network;
[0008] Step S3: Visible light images can provide rich color and texture information, while infrared images can provide better target detection and recognition capabilities at night or in adverse weather conditions. Therefore, fusing visible light and infrared images can improve image quality and enhance image target detection capabilities. First, the RGB image of visible light is converted to the YCrCb color space while retaining the chromaticity information. The Y channel is then fused. Then, the infrared image and the visible light brightness channel are classified and subjected to multi-level wavelet decomposition to obtain low-frequency approximation coefficients and high-frequency detail coefficients.
[0009] Step S4: The general target detection algorithm Yolov11 is combined with HTB to improve the impact of severe weather on the forward vehicle detection algorithm;
[0010] Step S5: Introduce DHSA into the Yolov11 backbone network to replace the traditional attention mechanism in PSA, enabling it to focus on regions with similar degradation patterns. The improved PSA is as follows: Figure 1 As shown, HTB is then introduced into the Neck module of Yolov11, which enables the prediction head to more effectively focus on important targets in the degraded region, thereby improving detection accuracy.
[0011] In the above technical solution, in step S4, the YOLOv11 includes a C2PSA module, which is an advanced module for enhancing feature extraction. It combines the CSP structure and the PSA attention mechanism to further enhance the feature extraction capability of the model.
[0012] In the above technical solution, further, the method for building the weather multi-label classification model in step S2 includes:
[0013] First, data augmentation and expansion are performed by improving mosaic and image synthesis techniques, and then a multi-label weather classification model is trained.
[0014] The model can identify the weather in real time as either normal or severe, and then label it as either day or night.
[0015] In the above technical solution, furthermore, in step S3, adaptive fusion is performed according to the following rules:
[0016] (1) The low-frequency coefficients are fused with an infrared weight of 0.8 and a visible weight of 0.2.
[0017] (2) The high-frequency coefficients are fused with infrared and visible light based on local energy calculation and adaptive weights. Then, the fused coefficients are reconstructed into an image. Finally, the fused Y channel and the original CrCb channel are merged and converted back to an RGB color image.
[0018] In the above technical solution, the HTB further includes two modules: Dynamic Range Histogram Self Attention (DHSA) and Dual Scale Gated Feedforward Network, which work together to extract features from weather-affected images.
[0019] In the above technical solution, further, in steps S1-S5, a detection model containing 8 categories is trained on the above-mentioned network structure of HTB and YOLOv11 based on the autonomous driving data BDD100, specifically: person, rider, car, bus, truck, bike, motor, and train, for a total of 8 categories for early warning.
[0020] Since forward collision warning only needs to focus on the vehicle in front and the vehicles on both sides within a certain range of the vehicle, the trapezoidal ROI area is first determined. The adaptive cruise control system generally needs to detect 50 meters, which corresponds to a height of 0.2Himage (Himage is the height of the image) and a bottom width of 1.5 lanes. Therefore, we use the positional relationship between the monocular camera and the vehicle and the above information as a reference to draw the ROI area with the image width as the center, determine the four points of the ROI area to obtain the trapezoidal detection area, and customize the ROI area.
[0021] The model detects many vehicles and uses the ROI region reduction algorithm, which checks whether the center point of the vehicle detection box is within the ROI region. If the center point is within the ROI region, the vehicle is close to the vehicle and is judged as a valid vehicle that needs a warning. If the center point is not within the ROI region, the vehicle is far away from the vehicle and is judged as an invalid vehicle that does not need a warning. This can filter out a large number of irrelevant vehicle detection boxes and obtain valid vehicle detection boxes that may need a warning.
[0022] There may be multiple vehicles that need to be warned, so the IOU matching algorithm is used for tracking in order to record the distance of each vehicle from the current vehicle. The IOU matching tracking algorithm is a tracking mode that uses the highly overlapping IOUs of an object detected in consecutive frames. Its IOU measurement method is shown in formula (1). The specific algorithm process is as follows: the effective vehicle detection box B of the current frame and all effective detection boxes of the previous frame are compared with the IOU. The maximum value is selected and it is greater than the threshold σIOU. Then it is judged to be the same target. The threshold σIOU is set to 0.9 after experimentation. The above operation is performed by traversing all effective vehicle detection boxes of the current frame.
[0023] Formula (1): In autonomous driving, transforming the image coordinate system (2D pixel coordinates) to the world coordinate system (3D physical coordinates) is a key step; assuming the ground is flat, the target object is tall, the camera intrinsic matrix K is as shown in formula (2), and the camera extrinsic parameters are known, the transformation formula from image coordinate points to world coordinate points is shown in formula (3);
[0024] Formula (2):
[0025] fx, fy: Focal length, in pixels;
[0026] cx, cy: Principal points (image centers);
[0027] Formula (3):
[0028]
[0029] (u, v) are image coordinate points, (Xw, Yw, Zw) are world coordinate points, R is the camera extrinsic parameter, a 3*3 rotation matrix, and t is the camera extrinsic parameter, a 3*1 translation matrix; the calculation of Zc is divided into two cases:
[0030] In the case of a monocular camera, the ground point satisfies Yc = -h:
[0031] Formula (4):
[0032] Formula (5): In the case of a dual-camera setup:
[0033] Formula (6): B: Baseline, the distance between the two cameras;
[0034] d: Parallax;
[0035] The dynamic distance between the vehicle in front and the vehicle itself is used to determine whether a warning is needed. If the distance is getting closer and the collision time is less than the threshold TTCthresh, then the vehicle is considered to need a warning. Therefore, we use the vertical distance D from the center point of the effective vehicle detection box to the bottom edge of the ROI as the basis for the warning. Assume that the effective vehicle detection boxes are B1, B2, ..., BN, and the distance memory is Q1, Q2, ..., QN. Set the distance memory length of each effective vehicle to L, which is also the number of frames in which the effective vehicle appears. First, the formula (3) converts the image coordinate points to the world coordinate points and calculates the actual physical vertical distance D1, D2, ..., DN of the effective vehicles B1, B2, ..., BN in the current frame. When the number of times the effective vehicle appears is less than L, the vertical distance is directly stored in the distance memory. When the number of times it appears is greater than L, the distance is dynamically stored according to the first-in-first-out principle.
[0036] Bi: Represents the detection box for the i-th valid vehicle;
[0037] Qi: represents the distance memory corresponding to the detection frame of the i-th valid vehicle, which is a queue that stores the actual physical distance, with a length of L, where L represents the number of frames in which the vehicle appears; the stored distances are Di_1, Di_2, ..., Di_L;
[0038] Obtain the minimum distance Di_min, the maximum distance Di_max in the distance memory Qi of the valid vehicle, and the frame rate f of the camera. According to formula (7), calculate the average speed Vi corresponding to the maximum change in the distance between the preceding vehicle and the host vehicle within the L-frame time window, which reflects the magnitude of the relative speed of the preceding vehicle. Based on the latest actual physical vertical distance Di_L and the average speed Vi of the valid vehicle, use formula (8) to obtain the time to collision TTCi. Vi is calculated based on historical extreme values, which can reduce noise interference, reduce TTC jitter, and make the prediction more stable. At the same time, through the distance comparison in step 12, dynamically judge the trend of distance increase or decrease. If most of the distances in the distance memory are greater than the current distance D, it means that the preceding vehicle is approaching, i.e., Vi < 0, otherwise it is moving away, Vi > 0. In this way, the signed relative speed is obtained.
[0039] Formula (7):
[0040] Formula (8):
[0041] After investigation, the reaction time of the driver to the object in front is about 1.5 s. Under normal circumstances, TTC > 5 s belongs to low risk, 3 s < TTC <= 5 s belongs to medium risk, and TTC <= 3 s belongs to high risk. At the same time, China's GB / T39901 stipulates that when the vehicle speed is 50 km / h, the AEB trigger time <= 1.6 s. Therefore, set the relative speed threshold Vthreth = 50 km / h. According to the results and speed of the weather classification model, it is divided into the following situations:
[0042] (1) Bad weather and relative speed Vi > Vthreth, TTCthreth is about 5 s;
[0043] (2) Bad weather and relative speed Vi <= Vthreth, TTCthreth is about 4.5 s;
[0044] (3) Sunny, daytime and relative speed Vi > Vthreth, TTCthreth is about 3 s;
[0045] (4) Sunny, daytime and relative speed Vi <= Vthreth, TTCthreth is about 2.5 s;
[0046] (5) Sunny, nighttime and relative speed Vi > Vthreth, TTCthreth is about 4 s;
[0047] (6) Sunny, nighttime and relative speed Vi <= Vthreth, TTCthreth is about 3.5 s;
[0048] At the same time, set the frame number ratio threshold ratio. For the effective vehicle Bi, compare the vertical distance D in the current frame with the values of all elements in the distance memory Qi of the vehicle. If ratio*L elements in Qi are greater than D and TTCi < TTCthreth, give a warning to the vehicle owner to maintain the distance; otherwise, if the conditions are not met, delete the element at the head of the distance memory and add the new element D to the tail; the setting of the distance memory and the frame number ratio threshold is to dynamically determine whether the vehicle in front is getting closer and closer, and the time to collision gives the state of the vehicle in front and the vehicle in the current frame.
[0049] In this technical solution,
[0050] The beneficial effects of the present invention are:
[0051] 1. The method for predicting the collision of the vehicle in front based on vehicle detection and dynamic distance comparison. Using this method, the early warning of the collision of the vehicle in front can be completed through the weather multi-label classification algorithm, the adaptive fusion algorithm of visible light image and infrared image based on wavelet transform, the object detection algorithm based on the fusion of HTB and Yolov11, the ROI region reduction algorithm, the IOU matching tracking algorithm, and the dynamic distance storage algorithm.
[0052] 2. The method for predicting the collision of the vehicle in front based on vehicle detection and dynamic distance comparison. These algorithm models are small, occupy less resources, and take less time, so the prediction of the collision of the vehicle in front can be completed under the limitations of mobile devices and the continuity of monitoring vehicles. Description of the Drawings
[0053] Figure 1 is the structural schematic diagram of the present invention;
[0054] Figure 2 is the structural schematic diagram of the present invention;
[0055] Figure 3 is the structural schematic diagram of the present invention. Detailed Embodiments
[0056] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope protected by the present application.
[0057] In the description of this application, it should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. For ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.
[0058] Example 1:
[0059] Please see Figures 1-3 As shown, this embodiment provides a forward collision prediction method based on vehicle detection and dynamic distance comparison.
[0060] Includes the following steps:
[0061] Step S1: First, train a weather classification model to identify the current weather conditions in real time. Then, adopt different image denoising methods to improve image quality for different weather conditions. After image preprocessing, adopt an improved target detection algorithm to detect road environment targets such as pedestrians and vehicles in front. Finally, give a response based on the target detection results using the dynamic distance between the vehicle and the vehicle in front.
[0062] Step S2: Build a weather multi-label classification model using MobileNet, a lightweight convolutional neural network;
[0063] Step S3: Visible light images can provide rich color and texture information, while infrared images can provide better target detection and recognition capabilities at night or in adverse weather conditions. Therefore, fusing visible light and infrared images can improve image quality and enhance image target detection capabilities. First, the RGB image of visible light is converted to the YCrCb color space while retaining the chromaticity information. The Y channel is then fused. Then, the infrared image and the visible light brightness channel are classified and subjected to multi-level wavelet decomposition to obtain low-frequency approximation coefficients and high-frequency detail coefficients.
[0064] Step S4: The general target detection algorithm Yolov11 is combined with HTB to improve the impact of severe weather on the forward vehicle detection algorithm;
[0065] Step S5: Introduce DHSA into the Yolov11 backbone network to replace the traditional attention mechanism in PSA, enabling it to focus on regions with similar degradation patterns. The improved PSA is as follows: Figure 1 As shown, HTB is then introduced into the Neck module of Yolov11, which enables the prediction head to more effectively focus on important targets in the degraded region, thereby improving detection accuracy.
[0066] Example 2:
[0067] This embodiment provides a forward collision prediction method based on vehicle detection and dynamic distance comparison. In addition to the technical solutions of the above embodiments, it also has the following technical features.
[0068] In step S4, the YOLOv11 includes a C2PSA module, which is a high-level module for enhancing feature extraction. It combines the CSP structure and the PSA attention mechanism to further enhance the model's feature extraction capabilities.
[0069] Example 3:
[0070] This embodiment provides a forward collision prediction method based on vehicle detection and dynamic distance comparison. In addition to the technical solutions of the above embodiments, it also has the following technical features.
[0071] The method for building the weather multi-label classification model in step S2 includes:
[0072] First, data augmentation and expansion are performed by improving mosaic and image synthesis techniques, and then a multi-label weather classification model is trained.
[0073] The model can identify the weather in real time as either normal or severe, and then label it as either day or night.
[0074] Example 4:
[0075] This embodiment provides a forward collision prediction method based on vehicle detection and dynamic distance comparison. In addition to the technical solutions of the above embodiments, it also has the following technical features.
[0076] In step S3, adaptive fusion is performed according to the following rules:
[0077] (1) The low-frequency coefficients are fused with an infrared weight of 0.8 and a visible weight of 0.2.
[0078] (2) The high-frequency coefficients are fused with infrared and visible light based on local energy calculation and adaptive weights. Then, the fused coefficients are reconstructed into an image. Finally, the fused Y channel and the original CrCb channel are merged and converted back to an RGB color image.
[0079] Example 5:
[0080] This embodiment provides a forward collision prediction method based on vehicle detection and dynamic distance comparison. In addition to the technical solutions of the above embodiments, it also has the following technical features.
[0081] The HTB consists of two modules: Dynamic Range Histogram Self Attention (DHSA) and Dual Scale Gated Feedforward Network, which work together to extract features from weather-affected images.
[0082] Example 6:
[0083] This embodiment provides a forward collision prediction method based on vehicle detection and dynamic distance comparison. In addition to the technical solutions of the above embodiments, it also has the following technical features.
[0084] In steps S1-S5, a detection model containing 8 categories is trained on the network structure of the above-mentioned HTB and Yolov11 fusion based on the autonomous driving data BDD100. Specifically, the model is used to issue warnings for 8 categories: person, rider, car, bus, truck, bike, motor, and train.
[0085] Since forward collision warning only needs to focus on the vehicle in front and the vehicles on both sides within a certain range of the vehicle, the trapezoidal ROI area is first determined. The adaptive cruise control system generally needs to detect 50 meters, which corresponds to a height of 0.2Himage (Himage is the height of the image) and a bottom width of 1.5 lanes. Therefore, we use the positional relationship between the monocular camera and the vehicle and the above information as a reference to draw the ROI area with the image width as the center, determine the four points of the ROI area to obtain the trapezoidal detection area, and customize the ROI area.
[0086] The model detects many vehicles and uses the ROI region reduction algorithm, which checks whether the center point of the vehicle detection box is within the ROI region. If the center point is within the ROI region, the vehicle is close to the vehicle and is judged as a valid vehicle that needs a warning. If the center point is not within the ROI region, the vehicle is far away from the vehicle and is judged as an invalid vehicle that does not need a warning. This can filter out a large number of irrelevant vehicle detection boxes and obtain valid vehicle detection boxes that may need a warning.
[0087] There may be multiple vehicles that need to be warned, so the IOU matching algorithm is used for tracking in order to record the distance of each vehicle from the current vehicle. The IOU matching tracking algorithm is a tracking mode that uses the highly overlapping IOUs of an object detected in consecutive frames. Its IOU measurement method is shown in formula (1). The specific algorithm process is as follows: the effective vehicle detection box B of the current frame and all effective detection boxes of the previous frame are compared with the IOU. The maximum value is selected and it is greater than the threshold σIOU. Then it is judged to be the same target. The threshold σIOU is set to 0.9 after experimentation. The above operation is performed by traversing all effective vehicle detection boxes of the current frame.
[0088] Formula (1):
[0089] In autonomous driving, transforming the image coordinate system (2D pixel coordinates) to the world coordinate system (3D physical coordinates) is a key step; assuming the ground is flat, the target object is tall, the camera intrinsic matrix K is as shown in formula (2), and the camera extrinsic parameters are known, the transformation formula from image coordinate points to world coordinate points is shown in formula (3);
[0090] Formula (2):
[0091] fx, fy: Focal length, in pixels;
[0092] cx, cy: Principal points (image centers);
[0093] Formula (3):
[0094] (u, v) are image coordinate points, (Xw, Yw, Zw) are world coordinate points, R is the camera extrinsic parameter, a 3*3 rotation matrix, and t is the camera extrinsic parameter, a 3*1 translation matrix; the calculation of Zc is divided into two cases:
[0095] In the case of a monocular camera, the ground point satisfies Yc = -h:
[0096] Formula (4):
[0097] Formula (5):
[0098] In the case of a dual-camera setup:
[0099] Formula (6):
[0100] B: Baseline, the distance between the two cameras;
[0101] d: Parallax;
[0102] The dynamic distance between the vehicle in front and the vehicle itself is used to determine whether a warning is needed. If the distance is getting closer and the collision time is less than the threshold TTCthresh, then the vehicle is considered to need a warning. Therefore, we use the vertical distance D from the center point of the effective vehicle detection box to the bottom edge of the ROI as the basis for the warning. Assume that the effective vehicle detection boxes are B1, B2, ..., BN, and the distance memory is Q1, Q2, ..., QN. Set the distance memory length of each effective vehicle to L, which is also the number of frames in which the effective vehicle appears. First, the formula (3) converts the image coordinate points to the world coordinate points and calculates the actual physical vertical distance D1, D2, ..., DN of the effective vehicles B1, B2, ..., BN in the current frame. When the number of times the effective vehicle appears is less than L, the vertical distance is directly stored in the distance memory. When the number of times it appears is greater than L, the distance is dynamically stored according to the first-in-first-out principle.
[0103] Bi: Represents the detection box for the i-th valid vehicle;
[0104] Qi: represents the distance memory corresponding to the detection frame of the i-th valid vehicle, which is a queue that stores the actual physical distance, with a length of L, where L represents the number of frames in which the vehicle appears; the stored distances are Di_1, Di_2, ..., Di_L;
[0105] The minimum distance Di_min, maximum distance Di_max in the distance memory Qi of the effective vehicle and the frame rate f of the camera are obtained. The average speed Vi corresponding to the maximum change in distance between the preceding vehicle and the current vehicle within the L-frame time window is calculated according to formula (7), which reflects the amplitude of the relative speed of the preceding vehicle. Based on the latest actual physical vertical distance Di_L and average speed Vi of the effective vehicle, the collision time TTCi is obtained using formula (8). Vi is calculated based on historical extreme values, which can reduce noise interference, reduce TTC jitter, and make the prediction more stable. At the same time, the distance increase and decrease trend is dynamically judged by the distance comparison in step 12. If most distances in the distance memory are greater than the current distance D, it means that the preceding vehicle is approaching, i.e., Vi<0. Conversely, if most distances are greater than the current distance D, it means that the preceding vehicle is moving away, i.e., Vi>0. In this way, the relative speed with a sign is obtained.
[0106] Formula (7):
[0107] Formula (8):
[0108] After research, the reaction time of drivers to objects ahead is about 1.5 s. Under normal circumstances, TTC > 5 s belongs to low risk, 3 s < TTC <= 5 s belongs to medium risk, and TTC <= 3 s belongs to high risk; at the same time, China's GB / T 39901 stipulates that when the vehicle speed is 50 km / h, the AEB trigger time <= 1.6 s; therefore, the relative speed threshold Vthreth = 50 km / h is set. According to the results and speeds of the weather classification model, it is divided into the following situations:
[0109] (1) Bad weather and relative speed Vi > Vthreth, TTCthreth is about 5 s;
[0110] (2) Bad weather and relative speed Vi <= Vthreth, TTCthreth is about 4.5 s;
[0111] (3) Sunny, daytime and relative speed Vi > Vthreth, TTCthreth is about 3 s;
[0112] (4) Sunny, daytime and relative speed Vi <= Vthreth, TTCthreth is about 2.5 s;
[0113] (5) Sunny, night and relative speed Vi > Vthreth, TTCthreth is about 4 s;
[0114] (6) Sunny, night and relative speed Vi <= Vthreth, TTCthreth is about 3.5 s;
[0115] At the same time, the frame number ratio threshold ratio is set. For the effective vehicle Bi, the vertical distance D in the current frame is compared with the values of all elements in the vehicle distance memory Qi. If ratio * L elements in Qi are greater than D, and TTCi < TTCthreth, a warning for the vehicle owner to keep a distance is given; otherwise, if the conditions are not met, the element at the head of the distance memory is deleted, and the new element D is added to the tail; the setting of the distance memory and the frame number ratio threshold is to dynamically judge whether the vehicle ahead is getting closer and closer, and the collision time gives the state of the vehicle ahead and the vehicle in the current frame.
[0116] The present invention: Using this method, it is possible to complete the early warning of collision with the vehicle ahead through the weather multi-label classification algorithm, the adaptive fusion algorithm of visible light images and infrared images based on wavelet transform, the object detection algorithm based on the fusion of HTB and Yolov11, the ROI region reduction algorithm, the IOU matching tracking algorithm, and the dynamic distance storage algorithm; these algorithm models are small, occupy less resources, and take less time, so it is possible to complete the prediction of collision with the vehicle ahead under the limitations of mobile devices and the continuity of monitored vehicles.
[0117] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A method for predicting forward collisions based on vehicle detection and dynamic distance comparison, characterized in that, Includes the following steps: Step S1: First, train a weather classification model to identify the current weather conditions in real time. Then, adopt different image denoising methods to improve image quality for different weather conditions. After image preprocessing, adopt an improved target detection algorithm to detect road environment targets such as pedestrians and vehicles in front. Finally, give a response based on the target detection results using the dynamic distance between the vehicle and the vehicle in front. Step S2: Build a weather multi-label classification model using MobileNet, a lightweight convolutional neural network; Step S3: Visible light images can provide rich color and texture information, while infrared images can provide better target detection and recognition capabilities at night or in adverse weather conditions. Therefore, fusing visible light and infrared images can improve image quality and enhance image target detection capabilities. First, the RGB image of visible light is converted to the YCrCb color space while retaining the chromaticity information. The Y channel is then fused. Then, the infrared image and the visible light brightness channel are classified and subjected to multi-level wavelet decomposition to obtain low-frequency approximation coefficients and high-frequency detail coefficients. Step S4: The general target detection algorithm Yolov11 is combined with HTB to improve the impact of severe weather on the forward vehicle detection algorithm; Step S5: Introduce DHSA into the backbone network of YOLOv11 to replace the traditional attention mechanism in PSA, enabling it to focus on regions with similar degradation patterns. The improved PSA is shown in Figure 1. Then, introduce HTB into the Neck module of YOLOv11, so that the prediction head can more effectively focus on important targets in the degradation region and improve detection accuracy.
2. The method for predicting forward collisions based on vehicle detection and dynamic distance comparison according to claim 1, characterized in that, In step S4, the YOLOv11 includes a C2PSA module, which is a high-level module for enhancing feature extraction. It combines the CSP structure and the PSA attention mechanism to further enhance the model's feature extraction capabilities.
3. The method for predicting forward collisions based on vehicle detection and dynamic distance comparison according to claim 1, characterized in that, The method for building the weather multi-label classification model in step S2 includes: First, data augmentation and expansion are performed by improving mosaic and image synthesis techniques, and then a multi-label weather classification model is trained. The model can identify the weather in real time as either normal or severe, and then label it as either day or night.
4. The method for predicting forward collisions based on vehicle detection and dynamic distance comparison according to claim 1, characterized in that, In step S3, adaptive fusion is performed according to the following rules: (1) The low-frequency coefficients are fused with an infrared weight of 0.8 and a visible weight of 0.
2. (2) The high-frequency coefficients are fused with infrared and visible light based on local energy calculation and adaptive weights. Then, the fused coefficients are reconstructed into an image. Finally, the fused Y channel and the original CrCb channel are merged and converted back to an RGB color image.
5. The method for predicting forward collisions based on vehicle detection and dynamic distance comparison according to claim 1, characterized in that, As stated above.
6. The method for predicting forward collisions based on vehicle detection and dynamic distance comparison according to claim 1, characterized in that, The HTB comprises two modules: Dynamic Range Histogram Self Attention (DHSA) and a Dual-Scale Gated Feedforward Network, which work together to extract features from weather-affected images.
7. The method for predicting forward collisions based on vehicle detection and dynamic distance comparison according to claim 1, characterized in that, In steps S1-S5, an 8-class detection model is trained on the network structure of the above-mentioned HTB and Yolov11 fusion based on the autonomous driving data BDD100. Specifically, the model is used to issue warnings for the 8 categories: person, rider, car, bus, truck, bike, motor, and train. Since forward collision warning only needs to focus on the vehicle in front and the vehicles on both sides within a certain range of the vehicle, the trapezoidal ROI area is first determined. The adaptive cruise control system generally needs to detect 50 meters, which corresponds to a height of 0.2Himage (Himage is the height of the image) and a bottom width of 1.5 lanes. Therefore, we use the positional relationship between the monocular camera and the vehicle and the above information as a reference to draw the ROI area with the image width as the center, determine the four points of the ROI area to obtain the trapezoidal detection area, and customize the ROI area. The model detects many vehicles and uses the ROI region reduction algorithm, which checks whether the center point of the vehicle detection box is within the ROI region. If the center point is within the ROI region, the vehicle is close to the vehicle and is judged as a valid vehicle that needs a warning. If the center point is not within the ROI region, the vehicle is far away from the vehicle and is judged as an invalid vehicle that does not need a warning. This can filter out a large number of irrelevant vehicle detection boxes and obtain valid vehicle detection boxes that may need a warning. There may be multiple vehicles that need to be warned, so the IOU matching algorithm is used for tracking in order to record the distance of each vehicle from the current vehicle. The IOU matching tracking algorithm is a tracking mode that uses the highly overlapping IOUs of an object detected in consecutive frames. Its IOU measurement method is shown in formula (1). The specific algorithm process is as follows: the effective vehicle detection box B of the current frame and all effective detection boxes of the previous frame are compared with the IOU. The maximum value is selected and it is greater than the threshold σIOU. Then it is judged to be the same target. The threshold σIOU is set to 0.9 after experimentation. The above operation is performed by traversing all effective vehicle detection boxes of the current frame. Formula (1): In autonomous driving, transforming the image coordinate system (2D pixel coordinates) to the world coordinate system (3D physical coordinates) is a key step; assuming the ground is flat, the target object is tall, the camera intrinsic matrix K is as shown in formula (2), and the camera extrinsic parameters are known, the transformation formula from image coordinate points to world coordinate points is shown in formula (3); Formula (2): fx, fy: Focal length, in pixels; cx, cy: Principal points (image centers); Formula (3): (u, v) are image coordinate points, (Xw, Yw, Zw) are world coordinate points, R is the camera extrinsic parameter, a 3*3 rotation matrix, and t is the camera extrinsic parameter, a 3*1 translation matrix; the calculation of Zc is divided into two cases: In the case of a monocular camera, the ground point satisfies Yc = -h: Formula (4): Formula (5): In the case of a dual-camera setup: Formula (6): B: Baseline, the distance between the two cameras; d: Parallax; Use the dynamic distance between the leading vehicle and the host vehicle to determine whether a warning is needed. If the distance is getting closer and the time to collision is less than the threshold TTCthresh, it is considered that the vehicle needs a warning. Therefore, we use the vertical distance D from the center point of the effective vehicle detection box to the bottom edge of the ROI as the warning basis. Assume the effective vehicle detection boxes are B1, B2, …, BN, and the distance memories are Q1, Q2, …, QN. Set the length of the distance memory for each effective vehicle to be L, and this length is also the number of frames when the effective vehicle appears. First, formula (3) converts the image coordinate points to world coordinate points, and calculates the actual physical vertical distances D1, D2, …, DN of the effective vehicles B1, B2, …, BN in the current frame. When the number of times the effective vehicle appears is less than L, directly store the vertical distance into the distance memory. When the number of times is greater than L, follow the first-in-first-out principle to dynamically store the distance. Bi: represents the detection box of the i-th effective vehicle; Qi: represents the distance memory corresponding to the detection box of the i-th effective vehicle, that is, the queue for storing the actual physical distance, with a length of L. At the same time, L represents the number of frames when the vehicle appears; the stored distances are Di_1, Di_2, …, Di_L; Obtain the minimum distance Di_min, maximum distance Di_max in the distance memory Qi of the effective vehicle and the frame rate f of the camera. According to formula (7), calculate the average speed Vi corresponding to the maximum change in the distance between the leading vehicle and the host vehicle within the L-frame time window, which reflects the magnitude of the relative speed of the leading vehicle. According to the latest actual physical vertical distance Di_L and the average speed Vi of the effective vehicle, use formula (8) to obtain the time to collision TTCi. Vi is calculated based on historical extreme values, which can reduce noise interference, reduce the jitter of TTC, and make the prediction more stable. At the same time, through the distance comparison in step 12, dynamically judge the trend of distance increase or decrease. Most of the distances in the distance memory are greater than the current distance D, indicating that the leading vehicle is approaching, that is, Vi < 0, otherwise it is moving away, Vi > 0. In this way, the signed relative speed is obtained. Formula (7): Formula (8): After investigation, the reaction time of the driver to the object in front is about 1.5 s. Under normal circumstances, TTC > 5 s belongs to low risk, 3 s < TTC <= 5 s belongs to medium risk, and TTC <= 3 s belongs to high risk; The relative speed threshold Vthreth = 50 km / h. According to the results and speed of the weather classification model, it is divided into the following situations: (1) Bad weather and relative speed Vi > Vthreth, TTCthreth is about 5 s; (2) Bad weather and relative speed Vi <= Vthreth, TTCthreth is about 4.5 s; (3) Sunny, daytime and relative speed Vi > Vthreth, TTCthreth is about 3 s; (4) Sunny, daytime and relative speed Vi <= Vthreth, TTCthreth is about 2.5 s; (5) Sunny, night and relative speed Vi > Vthreth, TTCthreth is about 4 s; (6) Sunny, night and relative speed Vi <= Vthreth, TTCthreth is about 3.5 s; Meanwhile, set the frame number ratio threshold ratio. For the valid vehicle Bi, compare the vertical distance D in the current frame with the values of all elements in the distance memory Qi of the vehicle. If ratio*L elements in Qi are greater than D and TTCi < TTCthreth, give a warning to the vehicle owner to maintain the distance; otherwise, if the conditions are not met, delete the element at the head of the distance memory and add the new element D to the tail; the settings of the distance memory and the frame number ratio threshold are for dynamically judging whether the vehicle in front is getting closer, and the time to collision gives the state of the vehicle in front and the associated state of the vehicle in the current frame.