Automatic driving target detection uncertainty method for improving expected function safety
By improving the YOLO object detector, cognitive and random uncertainty estimation methods are used to solve the problem of high uncertainty in severe weather conditions of the autonomous driving system, real-time and robust object detection are achieved, and automatic Safety and reliability of driving systems.
Patent Information
- Application Number
- CN202411451161.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-02-11
AI Technical Summary
In severe weather conditions, it is difficult for autonomous driving systems to accurately detect pedestrians, vehicles and obstacles in the surrounding environment, resulting in high uncertainty and increasing the risk of collision. The prior art methods such as MC-Dropout and Deep Ensemble have problems such as high computational cost, poor real-time performance and high complexity.
By improving the YOLO object detector, cognitive uncertainty and random uncertainty estimation methods were adopted, uncertainty estimation was performed using Gaussian mixed model and Laplace distribution model, and random uncertainty was included in the loss function. Perform calibration.
Real-time uncertainty estimation in severe weather conditions is achieved, reducing computing and storage overhead, enhancing the robustness of the model and the reliability and safety of the autonomous driving system.
Smart Images
Figure CN119408561A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method for improving the uncertainty of autonomous driving target detection for expected functional safety. Background Art
[0002] Object detection is an indispensable part of the autonomous driving perception system. Its performance directly affects the overall performance of the autonomous driving system and road safety. However, autonomous driving will face various complex driving environments such as heavy rain and heavy snow. In such bad weather, environmental visibility and sensor performance will be greatly reduced, resulting in blurred image details and target object edges. The object detection algorithm fails to correctly detect important information such as pedestrians, vehicles, and obstacles in the surrounding environment, or provides high confidence for the wrongly detected objects, making it impossible for the vehicle to make avoidance, braking actions or wrong decisions in time, thereby increasing the risk of collision with other vehicles or pedestrians. At this time, it is necessary to measure the degree of certainty of the output results of the object detection algorithm through uncertainty estimation to evaluate its reliability.
[0003] At present, the existing technologies have the following main shortcomings: 1. Uncertainty is not classified and processed. 2. Monte Carlo Dropout (MC-Dropout), Deep Ensemble and other methods are usually used to estimate the uncertainty of the target detection model results, but these methods have the following problems: The MC-Dropout method requires multiple forward propagations for the same input sample, which significantly increases the inference time and computing cost, and does not meet the real-time requirements of autonomous driving. In addition, since the dropout mode used in each inference is different, its output may have high variability, resulting in unstable results; the Deep Ensemble method requires training multiple independent models, each with its own parameter set, which increases the demand for computing and storage resources, and increases the complexity of algorithm deployment. Summary of the invention
[0004] In view of the shortcomings of the prior art, the present invention provides a method for improving the uncertainty of automatic driving target detection for the expected functional safety;
[0005] The method for improving the uncertainty of target detection for autonomous driving with expected functional safety includes the following steps:
[0006] Step 1: Use a camera to collect driving environment data on different roads and in different weather conditions. After the collected data is classified and labeled, it is divided into training set, validation set and test set according to the set ratio;
[0007] Step 2: Improve the existing YOLO target detector by estimating epistemic uncertainty and random uncertainty, and use random uncertainty to calibrate the confidence of the YOLO target detector;
[0008] Step 2.1: Estimating epistemic uncertainty for the YOLO object detector.
[0009] Step 2.1.1: Train the YOLO object detector D to locate and classify objects belonging to M known classes in the training set, generating an M-dimensional logit vector as follows:
[0010] l=(l1,...,l M ) T
[0011] Among them, l represents the logit vector of the current detected object, (l1,...,l M ) represents the original prediction score of the target detector for each category of the detected object, (.) T Represents the transpose of a vector;
[0012] Step 2.1.2: Define a Gaussian mixture model G to simulate the region of known categories in the logit space of the YOLO target detector D, as shown in the following formula:
[0013] G={G1,G2,...,G M}
[0014] For a known class i, i∈1,...,M, each G i is a Gaussian mixture model consisting of T components, where each individual component j is represented by the mean Covariance matrix and weight definition;
[0015] Step 2.1.3: Test the Gaussian mixture model G. Given a detected object and its logit vector l, calculate the log-likelihood estimate of the logit vector l under the Gaussian mixture model of each known category, as shown in the following formula:
[0016]
[0017] Among them, p(l;G i ) indicates that in the Gaussian mixture model G i The probability of observing vector l under is a mean Covariance matrix The probability density function of the multivariate Gaussian distribution;
[0018] Step 2.1.4: Calculate the log-likelihood vector P of the detected object, as shown below:
[0019] P=(log(p(l;G1)),...,log(p(l;G M )))
[0020] Where log(p(l;G i )) is the log-likelihood value of the detected object belonging to the Gaussian mixture model of category i. If the log-likelihood value is lower than the set threshold, it indicates that the match between the detected object and the known category is not high, and step 2.2 is executed for calibration. If the log-likelihood value is greater than or equal to the set threshold, it indicates that the match between the detected object and the known category is high, and step 3 is executed;
[0021] Step 2.2: Perform random uncertainty estimation and confidence calibration on the YOLO target detector;
[0022] Step 2.2.1: For the detected object input, use Laplace distribution modeling to predict the bounding box, including each value in the center point coordinates x, y, width w, and height h, and calculate the random uncertainty as shown in the following formula:
[0023]
[0024] in, is the probability density function of the Laplace distribution, is the predicted value of the center point coordinates and width and height of the predicted bounding box, It is the scale parameter of the predicted bounding box center point coordinates and width and height, that is, random uncertainty;
[0025] Step 2.2.2: Redesign the loss function of the YOLO target detector and add the random uncertainty obtained in step 2.2.1 as a regular term, as shown in the following formula:
[0026] Loss = L original +λ·L uncertainty
[0027] Loss represents the total loss of the target detector, L original is the original loss function of the target detector, L uncertainty is the regular term loss with random uncertainty added, λ is the weight of the regular term loss, and the regular term loss function including random uncertainty is defined as follows:
[0028]
[0029] Where Z is the number of samples in the training batch, Represents the scale parameters of the center point coordinates and width and height of the predicted bounding box of the zth sample;
[0030] Step 3: The data in the test set is detected by the calibrated YOLO target detector. The calibrated YOLO target detector directly outputs the bounding box, category label and calibrated confidence of each target.
[0031] The beneficial effects of adopting the above technical solution are:
[0032] The present invention provides a method for improving the uncertainty of autonomous driving target detection for expected functional safety. A real-time method is used to estimate the cognitive uncertainty and random uncertainty of the YOLO target detector, which solves the huge computational and storage overhead problems faced by current technologies when using MC-Dropout and Deep Ensemble methods to estimate uncertainty. While facilitating integrated deployment, it also meets the real-time requirements of the autonomous driving field. In addition, the present application incorporates random uncertainty estimation into the loss function, calibrates the model confidence through iterative learning, and enhances the robustness of the model, thereby improving the reliability and safety of the autonomous driving system in complex driving environments. It has the following beneficial effects:
[0033] 1. Dividing uncertainty into epistemic uncertainty and random uncertainty and using different methods to estimate these two types of uncertainty can more accurately understand the sources of uncertainty in the model and improve the transparency and interpretability of the model;
[0034] 2. Use a Gaussian mixture model to simulate the distribution of each known category in the target detector training data set in the logit space. When predicting, directly calculate the log likelihood to obtain cognitive uncertainty. This method does not require training additional models and only requires one forward propagation, so it does not increase significant storage and computing overhead.
[0035] 3. Use random uncertainty estimation results to calibrate the confidence of the target detector and enhance the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 4 is a flowchart of uncertainty estimation and confidence calibration in an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0038] The method for improving the uncertainty of target detection for autonomous driving with expected functional safety includes the following steps:
[0039] Step 1: Use a camera to collect driving environment data on different roads and in different weather conditions. After the collected data is classified and labeled, it is divided into training set, validation set and test set according to the set ratio;
[0040] In this embodiment, a GoPro 10Black camera was used to collect real vehicle data on different roads in Jiangsu Province, Zhejiang Province, Jilin Province, Liaoning Province and other regions in China, covering a variety of adverse weather scenes such as rain, snow, fog, etc. The collected data was classified and labeled (including road signs, traffic lights, pedestrians, vehicles, etc.) and divided into training set, validation set and test set in a ratio of 7:2:1;
[0041] Step 2: Improve the existing YOLO target detector by estimating cognitive uncertainty and random uncertainty, and use random uncertainty to calibrate the confidence of the YOLO target detector, as follows: Figure 1 As shown;
[0042] Step 2.1: Estimating epistemic uncertainty for the YOLO object detector.
[0043] Step 2.1.1: Train the YOLO object detector D to locate and classify objects belonging to M known classes in the training set, generating an M-dimensional logit vector as follows:
[0044] l=(l1,...,l M ) T
[0045] Among them, l represents the logit vector of the current detected object, (l1,...,l M ) represents the original prediction score of the target detector for each category of the detected object, (.) T Represents the transpose of a vector;
[0046] Step 2.1.2: Define a Gaussian mixture model G to simulate the region of known categories in the logit space of the YOLO target detector D, as shown in the following formula:
[0047] G={G1,G2,...,G M}
[0048] For a known class i, i∈1,...,M, each G i is a Gaussian mixture model consisting of T components, where each individual component j is represented by the mean Covariance matrix and weight definition;
[0049] Step 2.1.3: Test the Gaussian mixture model G. Given a detected object and its logit vector l, calculate the log-likelihood estimate of the logit vector l under the Gaussian mixture model of each known category, as shown in the following formula:
[0050]
[0051] Among them, p(l;G i ) indicates that in the Gaussian mixture model G i The probability of observing vector l under is a mean Covariance matrix The probability density function of the multivariate Gaussian distribution;
[0052] Step 2.1.4: Calculate the log-likelihood vector P of the detected object, as shown below:
[0053] P=(log(p(l;G1)),...,log(p(l;G M )))
[0054] Where log(p(l;G i )) is the log-likelihood value of the detected object belonging to the Gaussian mixture model of category i. The vector P contains the log-likelihood of the detected object in all known categories G1, G2, ..., G M The log-likelihood value under , thereby deriving the cognitive uncertainty of the detected object. If the log-likelihood value is lower than the set threshold, it indicates that the match between the detected object and the known category is not high, indicating that the uncertainty is high, and step 2.2 is performed for calibration. If the log-likelihood value is greater than or equal to the set threshold, it indicates that the match between the detected object and the known category is high, that is, it is determined that the detected object belongs to this category, so no calibration is required, and step 3 is performed;
[0055] Step 2.2: Perform random uncertainty estimation and confidence calibration on the YOLO target detector;
[0056] Step 2.2.1: For the detected object input, use Laplace distribution modeling to predict the bounding box, including each value in the center point coordinates x, y, width w, and height h, and calculate the random uncertainty as shown in the following formula:
[0057]
[0058] in, is the probability density function of the Laplace distribution, is the predicted value of the center point coordinates and width and height of the predicted bounding box, It is the scale parameter of the predicted bounding box center point coordinates and width and height, that is, random uncertainty;
[0059] Step 2.2.2: Redesign the loss function of the YOLO target detector and add the random uncertainty obtained in step 2.2.1 as a regular term. Incorporating uncertainty into the loss function can enable the target detector to learn a more accurate confidence, as shown in the following formula:
[0060] Loss = L original +λ·L uncertainty
[0061] Loss represents the total loss of the target detector, L original is the original loss function of the target detector, L uncertainty is the regularized loss with random uncertainty added, λ is the weight of the regularized loss, which is used to balance the contribution between the regularized loss and the original loss. The regularized loss function with random uncertainty is defined as follows:
[0062]
[0063] Where Z is the number of samples in the training batch, Represents the scale parameters of the center point coordinates and width and height of the predicted bounding box of the zth sample;
[0064] Step 3: The data in the test set is detected by the calibrated YOLO target detector. The calibrated YOLO target detector directly outputs the bounding box, category label and calibrated confidence of each target.
[0065] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.
Claims
1. A method for improving the uncertainty of automatic driving target detection for expected functional safety, characterized in that: The following steps are involved: Step 1: Use a camera to collect driving environment data on different roads and in different weather conditions. After the collected data is classified and labeled, it is divided into training set, validation set and test set according to the set ratio; Step 2: Improve the existing YOLO target detector by estimating epistemic uncertainty and random uncertainty, and use random uncertainty to calibrate the confidence of the YOLO target detector; Step 3: The data in the test set is detected by the calibrated YOLO target detector. The calibrated YOLO target detector directly outputs the bounding box, category label and calibrated confidence of each target.
2. The method for improving the uncertainty of automatic driving target detection for expected functional safety according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step 2.1: Estimating epistemic uncertainty for the YOLO object detector. Step 2.2: Perform random uncertainty estimation and confidence calibration for the YOLO object detector.
3. The method for improving the uncertainty of automatic driving target detection for expected functional safety according to claim 2, characterized in that: The step 2.1 specifically includes the following steps: Step 2.1.1: Train the YOLO object detector D to locate and classify objects belonging to M known classes in the training set, generating an M-dimensional logit vector as follows: l＝(l1,...,l M ) T Among them, l represents the logit vector of the current detected object, (l1,...,l M ) represents the original prediction score of the target detector for each category of the detected object, (.) T Represents the transpose of a vector; Step 2.1.2: Define a Gaussian mixture model G to simulate the region of known categories in the logit space of the YOLO target detector D, as shown in the following formula: G={G1,G2,...,G M } For a known class i, i∈1,...,M, each G i is a Gaussian mixture model consisting of T components, where each individual component j is represented by the mean Covariance matrix and weight definition; Step 2.1.3: Test the Gaussian mixture model G. Given a detected object and its logit vector l, calculate the log-likelihood estimate of the logit vector l under the Gaussian mixture model of each known category, as shown in the following formula: Among them, p(l;G i ) indicates that in the Gaussian mixture model G i The probability of observing vector l under is a mean Covariance matrix The probability density function of the multivariate Gaussian distribution; Step 2.1.4: Calculate the log-likelihood vector P of the detected object, as shown below: P=(log(p(l;G1)),...,log(p(l;G M ))) Where log(p(l;G i )) is the log-likelihood value under the Gaussian mixture model that the detected object belongs to category i. If the log-likelihood value is lower than the set threshold, it indicates that the match between the detected object and the known category is not high, and step 2.2 is performed for calibration. If the log-likelihood value is greater than or equal to the set threshold, it indicates that the match between the detected object and the known category is high, and step 3 is performed.
4. The method for improving the uncertainty of automatic driving target detection for expected functional safety according to claim 2, characterized in that: The step 2.2 specifically includes the following steps: Step 2.2.1: For the detected object input, use Laplace distribution modeling to predict the bounding box, including each value in the center point coordinates x, y, width w, and height h, and calculate the random uncertainty as shown in the following formula: in, is the probability density function of the Laplace distribution, is the predicted value of the center point coordinates and width and height of the predicted bounding box, It is the scale parameter of the predicted bounding box center point coordinates and width and height, that is, random uncertainty; Step 2.2.2: Redesign the loss function of the YOLO target detector and add the random uncertainty obtained in step 2.2.1 as a regular term, as shown in the following formula: Loss=L original +λ·L uncertainty Loss represents the total loss of the target detector, L original is the original loss function of the target detector, L uncertainty is the regular term loss with random uncertainty added, λ is the weight of the regular term loss, and the regular term loss function including random uncertainty is defined as follows: Where Z is the number of samples in the training batch, Represents the scale parameters of the center point coordinates and width and height of the predicted bounding box of the zth sample.