An uncertainty perception semi-supervised target detection method under automatic driving environment

Abstract

This invention discloses a semi-supervised object detection method based on uncertainty perception in an autonomous driving environment, comprising the following steps: S1, obtaining a teacher model with the ability to predict uncertainty through supervised learning with labeled data, utilizing data augmentation and an uncertainty loss function; S2, the teacher model generating fuzzy pseudo-labels using unlabeled data; S3, using the fuzzy pseudo-labels as the learning objective of the student model, quantifying the similarity of the uncertainty outputs of the teacher and student models. This method addresses the problem in existing semi-supervised object detection tasks of lacking an uncertainty mechanism to address the performance degradation caused by label scarcity.

Description

Technical Field

[0001] This invention belongs to the field of vehicle target recognition technology, specifically, it relates to a semi-supervised target detection method for uncertainty perception in an autonomous driving environment. Background Technology

[0002] In the field of object detection, cutting-edge methods primarily rely on supervised learning using deep learning models. However, when deep learning algorithms are deployed in autonomous driving, the model's performance heavily depends on training data with ground truth annotations, resulting in a relatively slow overall learning speed.

[0003] Chinese invention patent application CN202311138196.6 discloses a pre-training method for an autonomous driving perception model. The method includes: acquiring an existing point cloud dataset containing unlabeled data; labeling different categories of the unlabeled data using a pseudo-label generator combined with semi-supervised learning, and increasing the diversity of the point cloud dataset through wire harness resampling and object rescaling to obtain a unified dataset, wherein object rescaling enhances instance-level diversity, and wire harness resampling enhances scene-level diversity; pre-training a perception model using the unified dataset with a set overall loss function as the optimization objective; and performing classification perception on a target point cloud dataset for an autonomous driving task using the trained perception model. This invention improves the generalization ability of pre-training on downstream datasets and enhances the accuracy of autonomous driving.

[0004] The drawback of the aforementioned patent is that although the training perception model is used for classification perception, which enhances the accuracy of autonomous driving, the uncertainty of pseudo-labels in semi-supervised object detection tasks is ignored. This results in many false positive samples still existing in the pseudo-labels generated by inference, making it easy to overfit noisy labels with large uncertainties, causing model degradation and making it unable to learn diverse unlabeled data information well, leading to the loss of important information. Summary of the Invention

[0005] To address the problem of insufficient labeling leading to reduced algorithm detection performance in existing semi-supervised object detection tasks, this invention provides an uncertainty-aware semi-supervised object detection method for autonomous driving environments.

[0006] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0007] A semi-supervised target detection method based on uncertainty perception in an autonomous driving environment includes the following steps:

[0008] S1. Through supervised learning with labeled data, a teacher model with the ability to predict uncertainty is obtained by using data augmentation and uncertainty loss function;

[0009] S2. The teacher model uses unlabeled data to infer and generate fuzzy pseudo-labels;

[0010] S3. Using fuzzy pseudo-labels as the learning objective of the student model, the similarity of the uncertain outputs of the teacher and student models is quantified.

[0011] Furthermore, uncertainty is introduced into the model training process. The model parameters are treated as a parameter distribution, and the model likelihood is transformed into a Gaussian likelihood. Each output of the model is independently generated by the same probability distribution, as shown in the following formula:

[0012]

[0013] Where {x i ,y i} indicates that the input during the supervision process contains labeled samples, while {f w (x),σ} represents the uncertain results output by the teacher model;

[0014] The bounding box regression results obtained from the detection network are represented as [c x ,c y ,w,h]∈R 4 It mainly predicted the center coordinates of the bounding box and the size of the overall box, and encoded each regression result accordingly.

[0015] Model the uncertainty of the above regression results:

[0016] Assume that each parameter in the regression results follows a Gaussian distribution, and the variance of the prediction results is σ. 2 Based on the maximum likelihood estimation theory, the parameters of the detection model are optimized as shown in the following formula:

[0017]

[0018] Where ω represents the neural network model parameters.

[0019] ω* represents the optimized parameters, using the conditional probability maximization method (argmax).

[0020] y represents the output variable.

[0021] x represents the input variable.

[0022] p(y|x,w) represents the probability density function.

[0023] The first step is discretization, transforming it into a product form. Then, the natural logarithm is used to process the product form of the probability density function log(p(y|x,w)). Finally, a Gaussian probability density function is applied, where f(x) represents the mean of the model's predictions, and y... i For the true value of the sample, σ i 2 Let Variance be the variance.

[0024] Furthermore, the uncertainty loss function for teacher model training is designed as shown in the following formula:

[0025]

[0026] Uncertainty loss function L v l The calculation is of the true value y. i Compared with the predicted value f w (x i The sum of the differences between ) and the exponentially decaying weight exp(-α) i Adjust for the effect of each sample, where α i Indicates an uncertainty index;

[0027] Uncertainty loss function For the four bounding box coordinate parameters [Δc x ,Δc y The regression fitting of [Δw,Δh] consists of two parts: one is the residual regression term obtained by using random samples through the model, and the other is the regularization term that utilizes the uncertainty of the parameters.

[0028] Furthermore, the detailed steps of the teacher model inferring fuzzy pseudo-labels using unlabeled data include:

[0029] S201, Pseudo-labels and Self-training: Pseudo-label learning introduces a method for training deep neural networks in a semi-supervised manner. It enhances the learning process by generating surrogate labels on unlabeled data, using a variant of the prediction function or pseudo-label to directly infer and generate them without any supervised training. The surrogate labels are used together with labeled data as training targets to extract additional useful information, supplementing and improving the semi-supervised model training process.

[0030] S202. Define fuzzy pseudo-labels: Fuzzy pseudo-labels are obtained by directly inferring from the unlabeled data input to the teacher model. These fuzzy pseudo-labels also possess uncertainty. Uncertainty regression of the fuzzy pseudo-labels is expressed as: p(t) * |x)={t * ,α *}; where t * The bounding box coordinates of the fuzzy pseudo-label, α * It's about t* The specific parameters have the same source of uncertainty;

[0031] S203, Uncertainty Fusion Mechanism: The predicted regression uncertainty and positioning information are fused separately.

[0032] Furthermore, in the semi-supervised object detection task, a teacher-student model is used for self-training, which includes two stages: the student learns from the teacher and the teacher learns from the student's validation loss. The model uses the pseudo-labels inferred by the teacher model and its own ground truth labels to improve the learning quality of the student model, while updating the teacher model by evaluating the policy gradient calculated by the student model on the retained validation set.

[0033] Furthermore, the detailed steps for fusing the predicted regression uncertainty and location information are as follows:

[0034] The bounding box results are sorted from high to low confidence. These boxes are compared with the box with the highest confidence, and the IoU (Intersection over Union, generally referring to the intersection-over-union ratio between the model prediction and the true value) value is calculated.

[0035] Determine if the IoU between the two detection boxes is greater than the threshold; if it is, proceed to the next step.

[0036] Repeat the IoU calculation and comparison process until the detection box with the lowest confidence score is operated on.

[0037] Furthermore, the uncertainty fusion mechanism also includes the fusion and updating of fuzzy pseudo-labels from different iterative training stages; this is called external fusion.

[0038] The uncertainty fusion mechanism fuses the fuzzy pseudo-labels generated in the previous two iterations to obtain a more stable result, and repeats this update process until the training is completely finished.

[0039] Furthermore, the formula for calculating the similarity of the uncertain outputs of the teacher and student models is as follows:

[0040]

[0041] It is a holistic description of the loss functions of the teacher model and the student model in an unsupervised process;

[0042] The first part is the summation symbol ∑i=1N l It applies to i = 1 to N l The function L is used to compute each value of i. l (f t (x i l ,vt ,w t The values ​​of L are summed together. l This represents the loss function used by the teacher model to generate pseudo-labels during supervised learning, employing the cross-entropy loss function; where N... l Indicates the total number of labeled samples; f t Represents the teacher model; x i l v represents the i-th input sample for supervised learning; t This indicates the uncertainty of the teacher model; w t Indicates the teacher model weights;

[0043] Part 2 N u L represents the total number of unlabeled image samples. u This represents the unsupervised loss for unlabeled data, also using the pre-packaged cross-entropy loss function. * represents the pseudo-label inferred by the teacher model, γ is the weighting coefficient between supervised and unsupervised losses, u on x represents the unlabeled input, and v... s This represents the uncertainty of the student model; w s Indicates the student model weights;

[0044] By combining the uncertainty of the fuzzy pseudo-label data with the uncertainty of the student model, and based on the theory of KL divergence, the similarity of the output distributions of the two models is quantified as the optimization objective for secondary training.

[0045]

[0046] L KLD Let p represent the Kullback-Leibler divergence (KL divergence), an asymmetric measure of the difference between two probability distributions. t* (y|x) represents the probability density function of the teacher model for y given input x, p s (y|x) represents the probability density function of the student model for y given input x;

[0047] logp t* (y∣x) / p s (y|x) is the logarithmic ratio of two probabilities. It mainly compares the outputs of two probability functions. Then, the KL divergence is obtained by integrating the natural logarithm of the difference between the teacher model probability and the student model distribution. A KL divergence of zero indicates that the two distributions are exactly the same; a non-zero value indicates that there is a difference between them, and the larger the value, the more significant the difference.

[0048] The corresponding uncertainty loss function for unsupervised processes is designed as follows:

[0049]

[0050] in, It is a representation of relative uncertainty in unsupervised training, and combines the two variances of the output as the final description of uncertainty.

[0051] L v u This indicates that the uncertainty similarity between the teacher model and the student model is included in the loss function for evaluation. Where σ... s，j σ represents the uncertainty of the j-th output of the teacher model; t*，j Let f represent the uncertainty of the j-th output of the student model; multiplying the former by the latter raised to the power of -1 yields the relative uncertainty representation; t*，j (x) represents the output of the teacher model after unsupervised learning with input x; f s，j (x) represents the output of the student model with input x, and the square of the absolute value of the two.

[0052] Furthermore, the entire uncertainty-aware semi-supervised target detection method employs two data augmentation methods: elliptical rotation and random copying.

[0053] The shape of an object is fitted using an inscribed ellipse instead of a circumscribed rectangle.

[0054] After rotating the image, the bounding rectangle of the largest inscribed ellipse of the rotated bounding box is taken as the enhanced label. This largest inscribed ellipse is based on the bounding box positioning information [c x ,c y ,w,h]∈R 4 The specific calculation equation for generation is as follows:

[0055]

[0056] The above formula is one of the standard equations of an ellipse. It defines an ellipse centered at point (cx, cy), with a horizontal semi-axis of length 2w and a vertical semi-axis of length 2h. The maximum inscribed ellipse is calculated using the bounding box positioning information [cx, cy, w, h].

[0057] By dynamically optimizing and updating the IoU threshold through random rotation angles, the threshold requirements are met, and the next step of fusion and loss calculation is carried out according to the scheme described in the uncertainty fusion mechanism of fuzzy pseudo-labels.

[0058] The formula for updating the uncertainty threshold of IoU is:

[0059]

[0060] Thr(θ) represents the IoU uncertainty threshold obtained when the input is θ. θ0 is the "initial rotation angle parameter corresponding to the lowest IoU threshold of 0.5". When θ = θ0, Thr(θ) = 0.5. In the range 0-π / 2, θ is less than θ0. When θ is greater than 0, θ decreases, sin2θ decreases, and since the denominator is constant, the squared term decreases overall, and Thr(θ) increases.

[0061] Therefore, a smaller rotation angle will update a larger IoU threshold, and the probability of it being incorporated into the fuzzy pseudo-label is greater. This avoids the uncertainty in the label caused by large-angle rotation, thus adapting to larger-angle rotation enhancement.

[0062] Where θ is the rotation angle, and θ0 is the initial rotation angle parameter with a minimum IoU threshold of 0.5.

[0063] As can be seen, the updated IoU threshold can adapt to larger angle enhancements based on smaller rotation angles, and thus the model can also adapt to the uncertainty of blurred pseudo-labels caused by rotation enhancement to a certain extent.

[0064] Compared with the prior art, the present invention has the following advantages:

[0065] This invention employs an uncertainty-based semi-supervised object detection framework suitable for the field of autonomous driving. By introducing an uncertainty mechanism into the semi-supervised object detection task, it addresses the problem of reduced algorithm detection performance caused by label scarcity.

[0066] To address the characteristics of semi-supervised object detection tasks, we propose a general SSOD algorithm framework based on uncertainty. To our knowledge, this is the first time a training strategy based on two types of uncertainty has been proposed for a semi-supervised object detection task.

[0067] We propose an uncertainty fusion mechanism for pseudo-labels and the Fuzzy-NMS algorithm to utilize the training framework of an uncertain semi-supervised object detection model. This mechanism fuses the fuzzy pseudo-labels generated by multiple iterations of inference to improve their quality and optimize the training of student models.

[0068] At the data augmentation level, elliptical rotation is proposed and combined with random copying. By augmenting unlabeled data separately and fully incorporating the fuzzy pseudo-label information generated by inference, the model can obtain more stable and accurate output. Attached Figure Description

[0069] Figure 1 This is an overall flowchart of a semi-supervised target detection method for uncertainty perception in an autonomous driving environment, as described in an embodiment of the present invention.

[0070] Figure 2This is a flowchart illustrating the detailed steps of the teacher model in generating fuzzy pseudo-labels using unlabeled data in an embodiment of the present invention.

[0071] Figure 3 This is a flowchart of the uncertainty fusion mechanism in an embodiment of the present invention;

[0072] Figure 4 This is a visualization diagram showing the uncertainty of detection results in three typical complex autonomous driving scenarios in an embodiment of the present invention. Detailed Implementation

[0073] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0074] like Figure 1 As shown, this embodiment provides a semi-supervised target detection method based on uncertainty perception in an autonomous driving environment, including the following steps:

[0075] S1. Through supervised learning with labeled data, a teacher model with the ability to predict uncertainty is obtained by using data augmentation and uncertainty loss function;

[0076] S2. The teacher model uses unlabeled data to infer and generate fuzzy pseudo-labels;

[0077] S3. Using fuzzy pseudo-labels as the learning objective of the student model, the similarity of the uncertain outputs of the teacher and student models is quantified.

[0078] Uncertainty is introduced into model training, treating model parameters as a parameter distribution, and the model likelihood is transformed into Gaussian likelihood. Each output of the model is independently generated by the same probability distribution, as shown in the following formula:

[0079]

[0080] Where {x i ,y i} indicates that the input during the supervision process contains labeled samples, while {f w (x),σ} represents the uncertain results output by the teacher model;

[0081] The bounding box regression results obtained from the detection network are represented as [c x ,c y ,w,h]∈R 4 It mainly predicted the center coordinates of the bounding box and the size of the overall box, and encoded each regression result accordingly.

[0082] Model the uncertainty of the above regression results:

[0083] Assume that each parameter in the regression results follows a Gaussian distribution, and the variance of the prediction results is σ. 2 Based on the maximum likelihood estimation theory, the parameters of the detection model are optimized as shown in the following formula:

[0084]

[0085] Where ω represents the neural network model parameters.

[0086] ω* represents the optimized parameters, using the conditional probability maximization method (argmax).

[0087] y represents the output variable.

[0088] x represents the input variable.

[0089] p(y|x,w) represents the probability density function.

[0090] The first step is discretization, transforming it into a product form. Then, the natural logarithm is used to process the product form of the probability density function log(p(y|x,w)). Finally, a Gaussian probability density function is applied, where f(x) represents the mean of the model's predictions, and y... i For the true value of the sample, σ i 2 Let Variance be the variance.

[0091] Furthermore, the uncertainty loss function for teacher model training is designed as shown in the following formula:

[0092]

[0093] Uncertainty loss function L v l The calculation is of the true value y. i Compared with the predicted value f w (x i The sum of the differences between ) and the exponentially decaying weight exp(-α) i Adjust for the effect of each sample, where α i Indicates an uncertainty index;

[0094] Uncertainty loss function For the four bounding box coordinate parameters [Δc x ,Δc y The regression fitting of [Δw, Δh] consists of two parts: first, the residual regression term obtained by the model using random samples; and second, the regularization term utilizing the uncertainty of the parameters. The uncertainty regularization term is set to prevent the network's predicted data from having infinite variance, thereby ensuring the stability of gradient descent.

[0095] Incorporating the uncertainty characterization of logarithm αi =logσ 2 Compared to the original regression variance σ 2 It is numerically more stable, avoiding the possibility of the loss function being divided by zero. exp(-α) i As an exponential mapping, it is interpreted as a positive value, thus giving an effective value for uncertainty.

[0096] In supervised learning of the teacher model, the labeled data used for training are trained with definite values. Therefore, only the learning of the regression task needs to be supervised, and the uncertainty can be implicitly learned autonomously from the loss function.

[0097] This demonstrates that when the input data to the teacher model contains significant noise, the model will automatically optimize by using a larger regression variance to reduce the value of the residuals. During the optimization of the loss function, the teacher model autonomously learns about uncertainty during training, thereby obtaining its own assessment of the uncertainty level of the current detection result. If the model struggles to identify a particular sample, the loss function value decreases relatively slowly, similar to increasing the weight of difficult samples, making the model more focused on learning difficult samples, thus improving the overall performance of the algorithm.

[0098] like Figure 2 As shown, the detailed steps of the teacher model inference to generate fuzzy pseudo-labels using unlabeled data include:

[0099] S201, Pseudo-labels and Self-training: Pseudo-label learning introduces a method for training deep neural networks in a semi-supervised manner. It enhances the learning process by generating surrogate labels on unlabeled data, using a variant of the prediction function or pseudo-label to directly infer and generate them without any supervised training. The surrogate labels are used together with labeled data as training targets to extract additional useful information, supplementing and improving the semi-supervised model training process.

[0100] S202. Define fuzzy pseudo-labels: Fuzzy pseudo-labels are obtained by directly inferring from the unlabeled data input to the teacher model. These fuzzy pseudo-labels also possess uncertainty. Uncertainty regression of the fuzzy pseudo-labels is expressed as: p(t) * |x)={t * ,α *}; where t * The bounding box coordinates of the fuzzy pseudo-label, α * It's about t * The specific parameters have the same source of uncertainty;

[0101] like Figure 3 As shown, S203, uncertainty fusion mechanism: the predicted regression uncertainty and the positioning information are fused separately.

[0102] In the semi-supervised object detection task, a teacher-student model is used for self-training, which includes two stages: the student learns from the teacher and the teacher learns from the student's validation loss. The model uses the pseudo-labels inferred by the teacher model and its own ground truth label to improve the learning quality of the student model, while updating the teacher model by evaluating the policy gradient calculated by the student model on the retained validation set.

[0103] Detailed steps for fusing the predicted regression uncertainty and location information separately:

[0104] The bounding box results are sorted from high to low confidence. These boxes are compared with the box with the highest confidence, and the IoU (Intersection over Union, generally referring to the intersection-over-union ratio between the model prediction and the true value) value is calculated.

[0105] Determine if the IoU between the two detection boxes is greater than the threshold; if it is, proceed to the next step.

[0106] Repeat the IoU calculation and comparison process until the detection box with the lowest confidence score is operated on.

[0107] The updated position and uncertainty information after fusion are obtained using the Fuzzy-NMS algorithm based on the following formula:

[0108]

[0109]

[0110] The specific steps of the Fuzzy-NMS algorithm:

[0111]

[0112] The uncertainty fusion mechanism also includes the fusion and updating of fuzzy pseudo-labels from different iterative training stages; this is called external fusion.

[0113] The uncertainty fusion mechanism fuses the fuzzy pseudo-labels generated in the previous two iterations to obtain a more stable result, and repeats this update process until the training is completely finished.

[0114] Formula for calculating the similarity of outputs from teacher and student models with varying degrees of uncertainty:

[0115]

[0116] It is a holistic description of the loss functions of the teacher model and the student model in an unsupervised process;

[0117] The first part is the summation symbol ∑i=1N lIt applies to i = 1 to N l The function L is used to compute each value of i. l (f t (x i l ,v t ,w t The values ​​of L are summed together. l This represents the loss function used by the teacher model to generate pseudo-labels during supervised learning, employing the cross-entropy loss function; where N... l Indicates the total number of labeled samples; f t Represents the teacher model; x i l v represents the i-th input sample for supervised learning; t This indicates the uncertainty of the teacher model; w t Indicates the teacher model weights;

[0118] Part 2 N u L represents the total number of unlabeled image samples. u This represents the unsupervised loss for unlabeled data, also using the pre-packaged cross-entropy loss function. * represents the pseudo-label inferred by the teacher model, γ is the weighting coefficient between supervised and unsupervised losses, u on x represents the unlabeled input, and v... s This represents the uncertainty of the student model; w s Indicates the student model weights;

[0119] By combining the uncertainty of the fuzzy pseudo-label data with the uncertainty of the student model, and based on the theory of KL divergence, the similarity of the output distributions of the two models is quantified as the optimization objective for secondary training.

[0120]

[0121] L KLD Let p represent the Kullback-Leibler divergence (KL divergence), an asymmetric measure of the difference between two probability distributions. t* (y|x) represents the probability density function of the teacher model for y given input x, p s (y|x) represents the probability density function of the student model for y given input x;

[0122] logp t* (y∣x) / p s (y|x) is the logarithmic ratio of two probabilities. It mainly compares the outputs of two probability functions. Then, the KL divergence is obtained by integrating the natural logarithm of the difference between the teacher model probability and the student model distribution. A KL divergence of zero indicates that the two distributions are exactly the same; a non-zero value indicates that there is a difference between them, and the larger the value, the more significant the difference.

[0123] The corresponding uncertainty loss function for unsupervised processes is designed as follows:

[0124]

[0125] in, This represents the relative uncertainty in unsupervised training, combining the two variances of the output as the final description of uncertainty. It also indicates that the algorithm of this invention simultaneously models both accidental and cognitive uncertainties during unsupervised training.

[0126] L v u This indicates that the uncertainty similarity between the teacher model and the student model is included in the loss function for evaluation. Where σ... s，j σ represents the uncertainty of the j-th output of the teacher model; t*，j Let f represent the uncertainty of the j-th output of the student model; multiplying the former by the latter raised to the power of -1 yields the relative uncertainty representation; t*，j (x) represents the output of the teacher model after unsupervised learning with input x; f s，j (x) represents the output of the student model with input x, and the square of the absolute value of the two.

[0127] The uncertainty-aware semi-supervised target detection method employs two data augmentation methods: elliptical rotation and random copying.

[0128] The shape of an object is fitted using an inscribed ellipse instead of a circumscribed rectangle.

[0129] After rotating the image, the bounding rectangle of the largest inscribed ellipse of the rotated bounding box is taken as the enhanced label. This largest inscribed ellipse is based on the bounding box positioning information [c x ,c y ,w,h]∈R 4 The specific calculation equation for generation is as follows:

[0130]

[0131] The above formula is one of the standard equations of an ellipse. It defines an ellipse centered at point (cx, cy), with a horizontal semi-axis of length 2w and a vertical semi-axis of length 2h. The maximum inscribed ellipse is calculated using the bounding box positioning information [cx, cy, w, h].

[0132] By dynamically optimizing and updating the IoU threshold through random rotation angles, the threshold requirements are met, and the next step of fusion and loss calculation is carried out according to the scheme described in the uncertainty fusion mechanism of fuzzy pseudo-labels.

[0133] The formula for updating the uncertainty threshold of IoU is:

[0134]

[0135] Thr(θ) represents the IoU uncertainty threshold obtained when the input is θ, and θ0 is the "initial rotation angle parameter corresponding to the lowest IoU threshold of 0.5". When θ = θ0, Thr(θ) = 0.5. In the range of 0-π / 2, θ is less than θ0. When it is greater than 0, θ becomes smaller, sin2θ becomes smaller, and the denominator is a constant, so the square term becomes smaller overall, and Thr(θ) becomes larger.

[0136] Where θ is the rotation angle, and θ0 is the initial rotation angle parameter with a minimum IoU threshold of 0.5.

[0137] As can be seen, the updated IoU threshold can adapt to larger angle enhancements based on smaller rotation angles, and thus the model can also adapt to the uncertainty of blurred pseudo-labels caused by rotation enhancement to a certain extent.

[0138] For autonomous driving tasks, the ability to detect distant small targets from high-resolution scene images is essential for ensuring system safety. These distant small targets, such as distant vehicles or pedestrians, are difficult to distinguish with the naked eye in scene images, and instances are relatively few, causing the model to be biased in the detection of medium and large targets. In addition, distant small targets occupy less area, resulting in poor position generalization ability and a lack of background diversity. Generally speaking, smaller targets have fewer anchor box matches than larger targets, and the corresponding IoU value is also lower, which is not conducive to detection. Relatively speaking, increasing the number of anchor box matches, to some extent, means increasing the probability of detection. This invention uses a random copying method for small target enhancement, which can balance the quality of the detector on large and small objects. The core operation of random copying is to copy a small object and then paste it to other places on the image, thereby increasing the number of small targets in a single image.

[0139] Obscuring the scene

[0140] Figure 4 (a) is an occlusion scenario at a crossroads, where road conditions are relatively complex. It is evident that vehicles turning and going straight ahead cause visual occlusion, and the shadows of trees on both sides of the road also obscure the vehicles, resulting in unclear horizontal outlines of the target vehicles. The detection results intuitively show that the higher the degree of occlusion, the greater the localization uncertainty. With increasing training iterations, the uncertainty of the bounding boxes gradually decreases, the model's cognitive ability improves, and it is able to detect previously missed targets. The final detection results are basically consistent with the ground truth labels.

[0141] Small goal scenario

[0142] Figure 4(b) Scenes involving smaller targets. Vehicles closer in the straight line from the main vehicle's perspective have clear outlines, resulting in less uncertainty in bounding box regression; however, distant vehicles are too far from the main vehicle, causing visual blurring and thus greater uncertainty. For pedestrians and riders moving on the left side of the image, their small size and surrounding tree occlusion lead to significant uncertainty in the model's perception. Compared to the ground truth labels, the final detection results still show one small, occluded target missing.

[0143] Darker scenes

[0144] Figure 4 (c) shows a low-light experimental scenario. The target vehicles ahead of the main vehicle in its lane have clear outlines and are relatively close, resulting in relatively certain detections. In contrast, other vehicles parked on the right-hand sidewalk have higher uncertainty due to the overall dim lighting and tree obstruction. During the intermediate training iterations, false detections appeared on the right. Observing the surrounding environment of these false detections, it's likely that redundant detections were performed on other obstacles on the sidewalk. Ultimately, by utilizing the uncertainty and performing multiple iterations, a result similar to the ground truth label was achieved.

[0145] In summary, the model exhibits higher uncertainty in scene samples with occlusion, small targets, and low lighting. This uncertainty positively impacts the reliability of the model's detection results. Uncertainty quantification and fusion during iterative training ensures the SSOD algorithm's generalization performance across complex and diverse autonomous driving scenarios.

[0146] Compared with the prior art, the present invention has the following advantages:

[0147] This invention employs an uncertainty-based semi-supervised object detection framework suitable for the field of autonomous driving. By introducing an uncertainty mechanism into the semi-supervised object detection task, it addresses the problem of reduced algorithm detection performance caused by label scarcity.

[0148] To address the characteristics of semi-supervised object detection tasks, we propose a general SSOD algorithm framework based on uncertainty. To our knowledge, this is the first time a training strategy based on two types of uncertainty has been proposed for a semi-supervised object detection task.

[0149] We propose an uncertainty fusion mechanism for pseudo-labels and the Fuzzy-NMS algorithm to utilize the training framework of an uncertain semi-supervised object detection model. This mechanism fuses the fuzzy pseudo-labels generated by multiple iterations of inference to improve their quality and optimize the training of student models.

[0150] At the data augmentation level, elliptical rotation is proposed and combined with random copying. By augmenting unlabeled data separately and fully incorporating the fuzzy pseudo-label information generated by inference, the model can obtain more stable and accurate output.

[0151] The foregoing has provided a detailed description of a semi-supervised target detection method for uncertainty perception in an autonomous driving environment. The specific embodiments described are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A semi-supervised target detection method based on uncertainty perception in an autonomous driving environment, characterized in that, Including the following steps: S1. Supervised learning of labeled data, using data augmentation and uncertainty loss function to obtain a teacher model with the ability to predict uncertainty; S2. The teacher model uses unlabeled data to infer and generate fuzzy pseudo-labels; both labeled and unlabeled data are image data. S3. Using fuzzy pseudo-labels as the learning objective of the student model, the similarity of the uncertain outputs of the teacher and student models is quantified. For step S1, uncertainty is introduced in the teacher model training. The model parameters are treated as a parameter distribution, and the model likelihood is transformed into Gaussian likelihood. Each output of the model is independently generated by the same probability distribution, as shown in the following formula: in This indicates that the input during the supervision process includes labeled samples, while This indicates that the output of the teacher model contains uncertainties. The bounding box regression results obtained from the teacher model are expressed as follows: It mainly predicted the center coordinates of the bounding box and the size of the overall box, and encoded each regression result accordingly. Model the uncertainty of the above regression results: Assume that each parameter in the regression results follows a Gaussian distribution, and the variance of the prediction results is... Based on the maximum likelihood estimation theory, the parameters of the teacher model are optimized as shown in the following formula: ； Where ω represents the neural network model parameters. ω * This represents the optimized parameters, using the conditional probability maximized (argmax). y represents the output variable. x represents the input variable. p(y|x,w) represents the probability density function. The first step is discretization, transforming it into a product form. Then, the natural logarithm is used to process the product form's probability density function log(p(y|x,w)). Finally, a Gaussian probability density function is applied, where f(x) represents the mean of the teacher model's predictions, and y... i For the true value of the sample, σ i 2 For variance; The uncertainty loss function for teacher model training is designed as shown in the following formula: ； Uncertainty loss function L v l The calculation is the true value y of the sample. i Compared with the predicted value f w (x i The sum of the differences between ) and the exponentially decaying weight exp(−α) i Adjust for the effect of each sample, where α i Indicates an uncertainty index; Uncertainty loss function For the four bounding box coordinate parameters Regression fitting consists of two parts: one is the residual regression term obtained by using random samples through the teacher model, and the other is the regularization term that utilizes the uncertainty of the parameters. The detailed steps of the teacher model inference to generate fuzzy pseudo-labels using unlabeled data include: S201, Pseudo-labels and Self-training: Pseudo-label learning introduces a method for training deep neural networks in a semi-supervised manner. It enhances the learning process by generating surrogate labels on unlabeled data, using a variant of the prediction function or pseudo-label to directly infer and generate them without any supervised training. The surrogate labels are used together with labeled data as training targets to extract additional useful information, supplementing and improving the semi-supervised model training process. S202. Define fuzzy pseudo-labels: Fuzzy pseudo-labels are obtained by directly inferring from the unlabeled data input to the teacher model. These fuzzy pseudo-labels also possess uncertainty. Uncertainty regression analysis of the fuzzy pseudo-labels is expressed as follows: ;in, These are the bounding box coordinates of the fuzzy pseudo-label. It is about The specific parameters have the same source of uncertainty; S203, Uncertainty Fusion Mechanism: The predicted regression uncertainty and location information are fused separately; Formula for calculating the similarity of outputs from teacher and student models with varying degrees of uncertainty: It is a holistic description of the loss functions of the teacher model and the student model in an unsupervised process; The first part is the summation symbol. It applies to i=1 to N l The function L is used to calculate each value of i. l (f t (x i l ,v t ,w t The values ​​of L are summed together. l This represents the loss function used by the teacher model to generate pseudo-labels during supervised learning, employing the cross-entropy loss function; where N... l Indicates the total number of labeled samples; f t Represents the teacher model; x i l v represents the i-th input sample for supervised learning; t This indicates the uncertainty of the teacher model; w t Indicates the teacher model weights; Part 2 N u L represents the total number of unlabeled image samples. u This represents the unsupervised loss for unlabeled data, also using the pre-packaged cross-entropy loss function. * represents the pseudo-labels inferred by the teacher model, γ is the weighting coefficient between supervised and unsupervised losses, and x... i u The 'u' above represents unlabeled input, and the 'v' above represents unlabeled input. s This represents the uncertainty of the student model; w s Indicates the student model weights; By combining the uncertainty of fuzzy pseudo-label data with the uncertainty of the student model, and based on the theory of KL divergence, the similarity of the output distributions of the two models is quantified as the optimization objective for secondary training: ； L KLD Let p represent the Kullback-Leibler divergence, an asymmetric measure of the difference between two probability distributions. t* (y|x) represents the probability density function of the teacher model for y given input x, p s (y|x) represents the probability density function of the student model for y given input x; logp t* (y∣x) / p s (y|x) is the logarithmic ratio of two probabilities. It mainly compares the outputs of two probability functions. Then, the KL divergence is obtained by integrating the natural logarithm of the difference between the teacher model probability and the student model distribution. A KL divergence of zero indicates that the two distributions are exactly the same; a non-zero value indicates that there is a difference between them, and the larger the value, the more significant the difference. The corresponding uncertainty loss function for unsupervised processes is designed as follows: in, It is a representation of relative uncertainty in unsupervised training, and combines the two variances of the output as the final description of uncertainty; L v u This indicates that the uncertainty similarity between the teacher model and the student model is included in the loss function for evaluation; where σ s，j σ represents the uncertainty of the j-th output of the teacher model; t*，j Let f represent the uncertainty of the j-th output of the student model; multiplying the former by the latter raised to the power of -1 yields the relative uncertainty representation; t*，j (x) represents the output of the teacher model after unsupervised learning with input x; f s，j (x) represents the output of the student model with input x, and the square of the absolute value of the two is calculated. The uncertainty-aware semi-supervised target detection method employs two data augmentation methods: elliptical rotation and random copying. The shape of an object is fitted using an inscribed ellipse instead of a circumscribed rectangle. After rotating the image, the bounding rectangle of the largest inscribed ellipse of the rotated bounding box is taken as the enhanced label. The largest inscribed ellipse is based on the bounding box positioning information. The specific calculation equation for generation is as follows: ； The above formula is one of the standard equations of an ellipse, defining an ellipse with respect to a point (c). x ,c y An ellipse centered at [c], with a horizontal semi-axis of length 2w and a vertical semi-axis of length 2h, is located using bounding box information [c]. x c y Calculate the maximum inscribed ellipse of [w, h]. By dynamically optimizing and updating the IoU threshold through random rotation angles, the threshold requirements are met, and the next step of fusion and loss calculation is carried out according to the scheme described in the uncertainty fusion mechanism of fuzzy pseudo-labels. The formula for updating the uncertainty threshold of IoU is: ； Thr(θ) represents the IoU uncertainty threshold obtained when the input is θ, and θ0 is the "initial rotation angle parameter corresponding to the lowest IoU threshold of 0.5". When θ = θ0, Thr(θ) = 0.

5. In the range of 0-π / 2, θ is less than θ0. When it is greater than 0, θ becomes smaller, sin2θ becomes smaller, and the denominator is a constant, so the square term becomes smaller overall, and Thr(θ) becomes larger. Where θ is the rotation angle. This is the initial rotation angle parameter corresponding to the lowest IoU threshold, which is 0.

5.

2. The semi-supervised target detection method for uncertainty perception in an autonomous driving environment according to claim 1, characterized in that, In semi-supervised object detection tasks, a teacher-student model is used for self-training, which includes two stages: the student learns from the teacher and the teacher learns from the student's validation loss. The teacher-student model uses the pseudo-labels inferred by the teacher model and its own ground truth label to improve the learning quality of the student model, while updating the teacher model by evaluating the policy gradient calculated by the student model on the retained validation set.

3. The semi-supervised target detection method for uncertainty perception in an autonomous driving environment according to claim 2, characterized in that, Detailed steps for fusing the predicted regression uncertainty and location information separately: Sort the bounding box results from highest to lowest confidence, compare these boxes with the box with the highest confidence, and calculate the IoU value; Determine if the IoU between the two detection boxes is greater than the threshold; if it is, proceed to the next step. Repeat the IoU calculation and comparison process until the detection box with the lowest confidence score is operated on.

4. The semi-supervised target detection method for uncertainty perception in an autonomous driving environment according to claim 3, characterized in that, The uncertainty fusion mechanism also includes the fusion and updating of fuzzy pseudo-labels from different training iterations; The uncertainty fusion mechanism fuses the fuzzy pseudo-labels generated in the previous two iterations to obtain a more stable result, and repeats this update process until the training is completely finished.

Images