Uncertainty perception semi-supervised target detection method in automatic driving environment

Abstract

The invention discloses an uncertainty perception semi-supervised target detection method in an automatic driving environment, and the method comprises the steps: S1, obtaining a teacher model with the uncertainty prediction capability through the supervised learning of labeled data through data enhancement and an uncertainty loss function; s2, the teacher model generates a fuzzy pseudo-label by utilizing label-free data reasoning; and S3, quantizing the similarity of the uncertainty output of the teacher model and the student model by taking the fuzzy pseudo tag as a learning target of the student model. The problem that an uncertainty mechanism is not introduced in an existing semi-supervised target detection task to deal with the problem that the algorithm detection performance is reduced due to label deficiency is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle target recognition, and in particular, relates to an uncertainty-aware semi-supervised target detection method in an autonomous driving environment. Background Art

[0002] In the field of object detection, cutting-edge methods mainly rely on deep learning models for supervised learning. When deep learning algorithms are actually used in autonomous driving, the performance of the model is heavily dependent on training data with true value annotations, which results in a relatively slow overall learning speed.

[0003] In the Chinese public invention patent with application number CN202311138196.6, a pre-training method for autonomous driving perception models is disclosed. The method includes: obtaining an existing point cloud dataset, the point cloud dataset contains unlabeled data; for the unlabeled data, using a pseudo-label generator combined with semi-supervised learning to label different categories, and increasing the diversity of the point cloud dataset through beam resampling and object rescaling to obtain a unified dataset, wherein the object rescaling is used to enhance the diversity at the instance level, and the beam resampling is used to enhance the diversity at the scene level; using the unified dataset, with the set overall loss function as the optimization target, pre-training the perception model; for autonomous driving tasks, on the target point cloud dataset, using the trained perception model for classification perception. The present invention improves the generalization ability of pre-training in downstream datasets and enhances the accuracy of autonomous driving.

[0004] The defect of the above patent is that although the trained perception model is used for classification perception to enhance the accuracy of autonomous driving, the uncertainty of the pseudo-labels of the semi-supervised target detection task is ignored, so that there are still many false positive samples in the pseudo-labels generated by reasoning, which is easy to overfit the noise labels with large uncertainty, causing model degradation and inability to learn diverse unlabeled data information well, resulting in the loss of important information. Summary of the invention

[0005] In view of the problem that existing semi-supervised target detection tasks lack the introduction of uncertainty mechanism to deal with the reduced algorithm detection performance caused by lack of labels, the present invention provides an uncertainty-aware semi-supervised target detection method in an autonomous driving environment.

[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is as follows:

[0007] An uncertainty-aware semi-supervised target detection method in an autonomous driving environment comprises the following steps:

[0008] S1. Through supervised learning with labeled data, a teacher model with the ability to predict uncertainty is obtained using data augmentation and uncertainty loss function;

[0009] S2, the teacher model uses unlabeled data reasoning to generate fuzzy pseudo labels;

[0010] S3. Using fuzzy pseudo-labels as the learning objective of the student model, we quantify the similarity of the uncertainty outputs of the teacher and student models.

[0011] Furthermore, uncertainty is introduced in model training, model parameters are regarded as parameter distribution, and model likelihood is converted into Gaussian likelihood. Each output of the model is independently generated by the same probability distribution, as shown in the following formula:

[0012]

[0013] where {x i ,y i} represents the labeled samples input during supervision, and {f w (x),σ} represents the uncertain results output by the teacher model;

[0014] The bounding box regression result obtained by the detection network is expressed as [c x ,c y ,w,h]∈R 4 , mainly predicting the center coordinates of the bounding box and the size of the overall box, and encoding each regression result accordingly;

[0015] Model the uncertainty in the above regression results:

[0016] Assume that each parameter in the regression result follows a Gaussian distribution and the variance of the prediction result is σ 2 , according to the maximum likelihood estimation theory, the parameters of the detection model are optimized as shown in the following formula:

[0017]

[0018] Where ω represents the neural network model parameters,

[0019] ω* represents the optimized parameters, using the maximum conditional probability (argmax),

[0020] y represents the output variable,

[0021] x represents the input variable,

[0022] p(y|x,w) represents the probability density function,

[0023] First, we discretize it into a continuous multiplication form, then take the natural logarithm to process the probability density function log(p(y|x,w)) in the product form, and finally use the probability density function form of Gaussian distribution, where f(x) represents the mean predicted by the model, y i is the true value of the sample, σ i 2 is the variance.

[0024] Furthermore, the uncertainty loss function for teacher model training is designed as shown in the following formula:

[0025]

[0026] Uncertainty loss function L v l The actual value y is calculated i and the predicted value f w (x i ) and is weighted by exponential decay exp(-α i ) adjusts the influence of each sample, where α i represents the uncertainty index;

[0027] Uncertainty loss function For the four bounding box coordinate parameters [Δc x ,Δc y ,Δw,Δh] for regression fitting, which includes two parts: one is the residual regression term obtained by using random samples through the model, and the other is the regularization term using the uncertainty of the parameters.

[0028] Furthermore, the detailed steps of the teacher model using unlabeled data reasoning to generate fuzzy pseudo labels include:

[0029] S201, Pseudo-label and self-training: Pseudo-label learning introduces a method of training deep neural networks in a semi-supervised manner. It enhances the learning process by generating proxy labels on unlabeled data, and directly infers the generation using prediction functions or variants of pseudo-labels without any supervised training. It uses proxy labels together with labeled data as training targets to mine additional useful information, and supplement and improve the semi-supervised model training process.

[0030] S202, define fuzzy pseudo-labels: the teacher model directly infers the unlabeled data input to obtain fuzzy pseudo-labels, and the obtained fuzzy pseudo-labels also have uncertainty; the uncertainty regression of the fuzzy pseudo-labels is expressed as: p(t * |x)={t * ,α *}; where t * are the bounding box coordinates of the blurred pseudo-label, α * It is about* Homologous uncertainty of specific parameters;

[0031] S203, uncertainty fusion mechanism: fuse the predicted regression uncertainty and positioning information separately.

[0032] Furthermore, a teacher-student model is used for self-training in the semi-supervised object detection task, which includes two stages: the student learns from the teacher and the teacher learns from the student's verification loss. The model uses the pseudo labels inferred by the teacher model and its own true labels to improve the learning quality of the student model, and updates the teacher model by evaluating the policy gradient calculated by the student model on the retained validation set.

[0033] Furthermore, the detailed steps of fusing the predicted regression uncertainty and positioning information are as follows:

[0034] Sort the bounding box results from high to low confidence, compare these boxes with the box with the highest confidence, and calculate the IoU (Intersection over Union, generally refers to the intersection over union ratio between the model's prediction and the true value) value;

[0035] Determine whether the IoU of the two detection boxes is greater than the threshold. If it is greater than the threshold, proceed to the next step;

[0036] The IoU calculation and comparison process is repeated until the detection box with the lowest confidence score is operated on.

[0037] Furthermore, the uncertainty fusion mechanism also includes the fusion update of fuzzy pseudo-labels in different iterative training stages, which is called outer fusion.

[0038] The uncertainty fusion mechanism performs uncertainty fusion on the fuzzy pseudo labels generated by the previous and subsequent iterations to obtain a more stable result, and repeats this updating process until the training is completely completed.

[0039] Furthermore, the similarity calculation formula for quantifying the uncertainty output of the teacher and student models is:

[0040]

[0041] It is an overall description of the loss function of the teacher model and the student model in the unsupervised process;

[0042] The first part is the summation symbol ∑i=1N l , it is from i = 1 to N l For each value i, the function L is calculated l (f t (x i l ,vt ,w t )) and add these values ​​together, L l Represents the loss function of the teacher model for supervised learning to generate pseudo labels, using the cross entropy loss function; where N l represents the total number of labeled samples; f t represents the teacher model; x i l represents the i-th input sample of supervised learning; v t represents the uncertainty of the teacher model; w t represents the teacher model weight;

[0043] Part II u is the total number of unlabeled image samples, L u represents the unsupervised loss of unlabeled data, and also uses the packaged cross entropy loss function. * represents the pseudo label inferred by the teacher model, γ is the weight coefficient between supervised and unsupervised losses, u on x represents unlabeled input, and v s represents the uncertainty of the student model; w s represents the student model weight;

[0044] Combining the data uncertainty of the fuzzy pseudo-label and the uncertainty of the student model, based on the theory of KL divergence, the similarity of the output distribution of the two models is quantified as the optimization goal for secondary training:

[0045]

[0046] L KLD represents the Kullback-Leibler divergence (KL divergence), an asymmetric measure of the difference between two probability distributions, where p t* (y|x) represents the probability density function of the teacher model for y given the input x, p s (y|x) represents the probability density function of the student model for y given the input x;

[0047] logp t* (y|x) / p s (y|x) is the logarithmic ratio of the two probabilities. It mainly compares the outputs of the two probability functions, and then integrates the natural logarithm of the teacher model probability multiplied by its difference from the student model distribution to obtain the KL divergence. A KL divergence of zero indicates that the two distributions are exactly the same; a non-zero value indicates that there is a difference between them, and the larger the value, the more significant the difference;

[0048] The corresponding uncertainty loss function in the unsupervised process is designed as follows:

[0049]

[0050] in, It is a relative uncertainty representation in unsupervised training, combining the two variances of the output as the final description of uncertainty.

[0051] L v u Indicates that the uncertainty similarity between the teacher model and the student model is included in the loss function for evaluation. s，j represents the uncertainty of the j-th output of the teacher model; σ t*，j represents the uncertainty of the j-th output of the student model; the former is multiplied by the latter to the power of -1 to obtain the relative uncertainty representation; f t*，j (x) represents the output of the teacher model after unsupervised learning under the input x; f s，j (x) represents the output of the student model under the input x, and the absolute value of the two is calculated.

[0052] Furthermore, the entire uncertainty-aware semi-supervised target detection method uses two data enhancement methods, ellipse rotation and random replication.

[0053] Use an inscribed ellipse instead of a circumscribed rectangle to fit the shape of the object;

[0054] After rotating the image, the circumscribed rectangle of the largest inscribed ellipse of the rotated bounding box is taken as the enhanced label. This largest inscribed ellipse is based on the bounding box positioning information [c x ,c y ,w,h]∈R 4 Generate, the specific calculation equation is:

[0055]

[0056] The above formula is one of the standard equations of an ellipse, which defines an ellipse centered at point (cx, cy) with a horizontal semi-axis length of 2w and a vertical semi-axis length of 2h. Its maximum inscribed ellipse is calculated using the bounding box positioning information [cx, cy, w, h].

[0057] The IoU threshold is dynamically optimized and updated by randomly rotating the angle to meet the threshold requirement, and the next step of fusion and loss calculation is performed according to the scheme described in the uncertainty fusion mechanism of fuzzy pseudo-labels;

[0058] The uncertainty threshold update formula of IoU is:

[0059]

[0060] Thr(θ) represents the IoU uncertainty threshold obtained when the input is θ. 0is the initial rotation angle parameter corresponding to the minimum IoU threshold of 0.5. When θ = θ 0 When Thr(θ)=0.5. At 0-π / 2, θ is less than θ 0 , when it is greater than 0, θ becomes smaller, sin2θ becomes smaller, and the denominator is a constant, so the square term becomes smaller as a whole, and Thr(θ) becomes larger.

[0061] Therefore, a smaller rotation angle will update a larger IoU threshold, and the probability of being integrated into the fuzzy pseudo-label is greater, thus avoiding the uncertainty in the label caused by large-angle rotation, thereby adapting to larger-angle rotation enhancement.

[0062] where θ is the rotation angle, θ 0 It is the initial rotation angle parameter corresponding to the minimum IoU threshold of 0.5.

[0063] It can be seen that the updated IoU threshold can adapt to the enhancement of larger angles based on smaller rotation angles, and thus the model can also adapt to the uncertainty of the fuzzy pseudo-labels caused by rotation enhancement to a certain extent.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] The present invention adopts an uncertainty-based semi-supervised target detection framework suitable for the field of autonomous driving, and addresses the problem of reduced algorithm detection performance caused by lack of labels by introducing an uncertainty mechanism into the semi-supervised target detection task.

[0066] Aiming at the characteristics of semi-supervised object detection tasks, a general SSOD algorithm framework based on uncertainty is proposed. As far as we know, this is the first time that a training strategy based on two uncertainties is proposed for semi-supervised object detection tasks.

[0067] Using the uncertain semi-supervised target detection model training framework, an uncertainty fusion mechanism of pseudo labels and the Fuzzy-NMS algorithm are proposed to fuse the fuzzy pseudo labels generated by multiple iterative reasoning to improve their quality and optimize the training of the student model.

[0068] At the data enhancement level, elliptical rotation is proposed and combined with random replication. By enhancing the unlabeled data separately and fully combining the fuzzy pseudo-label information generated by reasoning, the model can obtain more stable and accurate output. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is an overall flow chart of an uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to an embodiment of the present invention;

[0070] Figure 2A flowchart of the detailed steps of the teacher model in the embodiment of the present invention using unlabeled data reasoning to generate fuzzy pseudo labels;

[0071] Figure 3 This is a flow chart of the uncertainty fusion mechanism in an embodiment of the present invention;

[0072] Figure 4 Schematic diagram of uncertainty visualization of detection results of three typical complex autonomous driving scenarios in an embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0074] like Figure 1 As shown, this embodiment provides an uncertainty-aware semi-supervised target detection method in an autonomous driving environment, comprising the steps of:

[0075] S1. Through supervised learning with labeled data, a teacher model with the ability to predict uncertainty is obtained using data augmentation and uncertainty loss function;

[0076] S2, the teacher model uses unlabeled data reasoning to generate fuzzy pseudo labels;

[0077] S3. Using fuzzy pseudo-labels as the learning objective of the student model, we quantify the similarity of the uncertainty outputs of the teacher and student models.

[0078] Uncertainty is introduced in model training, model parameters are regarded as parameter distribution, model likelihood is converted into Gaussian likelihood, and each output of the model is independently generated by the same probability distribution, as shown in the following formula:

[0079]

[0080] where {x i ,y i} represents the labeled samples input during supervision, and {f w (x),σ} represents the uncertain results output by the teacher model;

[0081] The bounding box regression result obtained by the detection network is expressed as [c x ,c y ,w,h]∈R 4 , mainly predicting the center coordinates of the bounding box and the size of the overall box, and encoding each regression result accordingly;

[0082] Model the uncertainty in the above regression results:

[0083] Assume that each parameter in the regression result follows a Gaussian distribution and the variance of the prediction result is σ 2 , according to the maximum likelihood estimation theory, the parameters of the detection model are optimized as shown in the following formula:

[0084]

[0085] Where ω represents the neural network model parameters,

[0086] ω* represents the optimized parameters, using the maximum conditional probability (argmax),

[0087] y represents the output variable,

[0088] x represents the input variable,

[0089] p(y|x,w) represents the probability density function,

[0090] First, we discretize it into a continuous multiplication form, then take the natural logarithm to process the probability density function log(p(y|x,w)) in the product form, and finally use the probability density function form of Gaussian distribution, where f(x) represents the mean predicted by the model, y i is the true value of the sample, σ i 2 is the variance.

[0091] Furthermore, the uncertainty loss function for teacher model training is designed as shown in the following formula:

[0092]

[0093] Uncertainty loss function L v l The actual value y is calculated i and the predicted value f w (x i ) and is weighted by exponential decay exp(-α i ) adjusts the influence of each sample, where α i represents the uncertainty index;

[0094] Uncertainty loss function For the four bounding box coordinate parameters [Δc x ,Δc y ,Δw,Δh] for regression fitting, which includes two parts: one is the residual regression term obtained by using random samples through the model, and the other is the regularization term using the uncertainty of the parameters. The uncertainty regularization term is set to prevent the network from predicting data with infinite variance, thereby ensuring the stability of gradient descent.

[0095] Add logarithmic uncertainty representation αi =logσ 2 Than the original regression variance σ 2 It is numerically more stable, as it avoids the loss function from being divided by zero. i ) is interpreted as an exponential map to positive values, giving effective values ​​for the uncertainties.

[0096] In the supervised learning of the teacher model, the labeled data involved in the training is itself trained with certain values. Therefore, only the learning of the supervised regression task is required to implicitly learn the uncertainty from the loss function.

[0097] It can be seen that when the input data of the teacher model contains large noise, the model will optimize a larger regression variance to reduce the value of the residual. In the process of optimizing the loss function, the teacher model autonomously learns uncertainty during training, thereby obtaining the model's own assessment of the uncertainty of the current detection result. If the model has difficulty identifying a certain sample, the loss function value decreases slowly, which is similar to increasing the model's weight on difficult samples, making the model more focused on learning difficult samples, thereby improving the overall performance of the algorithm.

[0098] like Figure 2 As shown in Figure 1, the detailed steps of the teacher model using unlabeled data reasoning to generate fuzzy pseudo labels include:

[0099] S201, Pseudo-label and self-training: Pseudo-label learning introduces a method of training deep neural networks in a semi-supervised manner. It enhances the learning process by generating proxy labels on unlabeled data, and directly infers the generation using prediction functions or variants of pseudo-labels without any supervised training. It uses proxy labels together with labeled data as training targets to mine additional useful information, and supplement and improve the semi-supervised model training process.

[0100] S202, define fuzzy pseudo-labels: the teacher model directly infers the unlabeled data input to obtain fuzzy pseudo-labels, and the obtained fuzzy pseudo-labels also have uncertainty; the uncertainty regression of the fuzzy pseudo-labels is expressed as: p(t * |x)={t * ,α *}; where t * are the bounding box coordinates of the blurred pseudo-label, α * It is about * Homologous uncertainty of specific parameters;

[0101] like Figure 3 As shown, S203, uncertainty fusion mechanism: the predicted regression uncertainty and positioning information are fused separately.

[0102] In the semi-supervised object detection task, a teacher-student model is used for self-training, which includes two stages: the student learns from the teacher and the teacher learns from the student's verification loss. The model uses the pseudo labels inferred by the teacher model and its own true labels to improve the learning quality of the student model, and updates the teacher model by evaluating the policy gradient calculated by the student model on the retained validation set.

[0103] Detailed steps to fuse the predicted regression uncertainty and positioning information separately:

[0104] Sort the bounding box results from high to low confidence, compare these boxes with the box with the highest confidence, and calculate the IoU (Intersection over Union, generally refers to the intersection over union ratio between the model's prediction and the true value) value;

[0105] Determine whether the IoU of the two detection boxes is greater than the threshold. If it is greater than the threshold, proceed to the next step;

[0106] The IoU calculation and comparison process is repeated until the detection box with the lowest confidence score is operated on.

[0107] The updated position and uncertainty information after fusion is obtained through the Fuzzy-NMS algorithm based on the following formula:

[0108]

[0109]

[0110] The specific steps of the Fuzzy-NMS algorithm are:

[0111]

[0112] The uncertainty fusion mechanism also includes the fusion update of fuzzy pseudo-labels in different iterative training stages, which is called outer fusion.

[0113] The uncertainty fusion mechanism performs uncertainty fusion on the fuzzy pseudo labels generated by the previous and subsequent iterations to obtain a more stable result, and repeats this updating process until the training is completely completed.

[0114] The similarity calculation formula for quantifying the uncertainty output of the teacher and student models is:

[0115]

[0116] It is an overall description of the loss function of the teacher model and the student model in the unsupervised process;

[0117] The first part is the summation symbol ∑i=1N l, it is from i = 1 to N l For each value i, the function L is calculated l (f t (x i l ,v t ,w t )) and add these values ​​together, L l Represents the loss function of the teacher model for supervised learning to generate pseudo labels, using the cross entropy loss function; where N l represents the total number of labeled samples; f t represents the teacher model; x i l represents the i-th input sample of supervised learning; v t represents the uncertainty of the teacher model; w t represents the teacher model weight;

[0118] Part II u is the total number of unlabeled image samples, L u represents the unsupervised loss of unlabeled data, and also uses the packaged cross entropy loss function. * represents the pseudo label inferred by the teacher model, γ is the weight coefficient between supervised and unsupervised losses, u on x represents unlabeled input, and v s represents the uncertainty of the student model; w s represents the student model weight;

[0119] Combining the data uncertainty of the fuzzy pseudo-label and the uncertainty of the student model, based on the theory of KL divergence, the similarity of the output distribution of the two models is quantified as the optimization goal for secondary training:

[0120]

[0121] L KLD represents the Kullback-Leibler divergence (KL divergence), an asymmetric measure of the difference between two probability distributions, where p t* (y|x) represents the probability density function of the teacher model for y given the input x, p s (y|x) represents the probability density function of the student model for y given the input x;

[0122] logp t* (y|x) / p s (y|x) is the logarithmic ratio of the two probabilities. It mainly compares the outputs of the two probability functions, and then integrates the natural logarithm of the teacher model probability multiplied by its difference from the student model distribution to obtain the KL divergence. A KL divergence of zero indicates that the two distributions are exactly the same; a non-zero value indicates that there is a difference between them, and the larger the value, the more significant the difference;

[0123] The corresponding uncertainty loss function in the unsupervised process is designed as follows:

[0124]

[0125] in, It is a relative uncertainty representation in unsupervised training, and the two variances of the output are combined as the final description of uncertainty. It also shows that the algorithm of the present invention models both accidental and cognitive uncertainties in the unsupervised training process.

[0126] L v u Indicates that the uncertainty similarity between the teacher model and the student model is included in the loss function for evaluation. s，j represents the uncertainty of the j-th output of the teacher model; σ t*，j represents the uncertainty of the j-th output of the student model; the former is multiplied by the latter to the power of -1 to obtain the relative uncertainty representation; f t*，j (x) represents the output of the teacher model after unsupervised learning under the input x; f s，j (x) represents the output of the student model under the input x, and the absolute value of the two is calculated.

[0127] The entire uncertainty-aware semi-supervised target detection method uses two data enhancement methods: ellipse rotation and random replication.

[0128] Use an inscribed ellipse instead of a circumscribed rectangle to fit the shape of the object;

[0129] After rotating the image, the circumscribed rectangle of the largest inscribed ellipse of the rotated bounding box is taken as the enhanced label. This largest inscribed ellipse is based on the bounding box positioning information [c x ,c y ,w,h]∈R 4 Generate, the specific calculation equation is:

[0130]

[0131] The above formula is one of the standard equations of an ellipse, which defines an ellipse centered at point (cx, cy) with a horizontal semi-axis length of 2w and a vertical semi-axis length of 2h. Its maximum inscribed ellipse is calculated using the bounding box positioning information [cx, cy, w, h].

[0132] The IoU threshold is dynamically optimized and updated by randomly rotating the angle to meet the threshold requirement, and the next step of fusion and loss calculation is performed according to the scheme described in the uncertainty fusion mechanism of fuzzy pseudo-labels;

[0133] The uncertainty threshold update formula of IoU is:

[0134]

[0135] Thr(θ) represents the IoU uncertainty threshold obtained when the input is θ, θ 0 is the initial rotation angle parameter corresponding to the minimum IoU threshold of 0.5. When θ = θ 0 When Thr(θ)=0.5; at 0-π / 2, θ is less than θ 0 , when it is greater than 0, θ becomes smaller, sin2θ becomes smaller, and the denominator is a constant, so the square term becomes smaller overall and Thr(θ) becomes larger;

[0136] where θ is the rotation angle, θ 0 It is the initial rotation angle parameter corresponding to the minimum IoU threshold of 0.5.

[0137] It can be seen that the updated IoU threshold can adapt to the enhancement of larger angles based on smaller rotation angles, and thus the model can also adapt to the uncertainty of the fuzzy pseudo-labels caused by rotation enhancement to a certain extent.

[0138] For autonomous driving tasks, it is necessary to detect distant small targets from high-resolution scene images to ensure the safety of the system. These distant small targets, such as distant vehicles or pedestrians, are difficult to distinguish with the naked eye on the scene image, and there are relatively few examples, which makes the model biased in the detection of medium and large targets. In addition, distant small targets occupy less area and have poor position generalization ability, resulting in a lack of background diversity. Generally speaking, smaller targets have fewer anchor frame matches than larger targets, and the corresponding IoU values ​​are also lower, which is not conducive to detection. Relatively speaking, increasing the number of anchor frame matches means increasing the probability of being detected to some extent. The present invention uses a random copy method to enhance small targets, which can weigh the quality of the detector on large objects and small objects. The core operation of random copying is to copy a small object and then paste it to other places on the picture, thereby increasing the number of small targets on a single picture.

[0139] Occlusion scene

[0140] Figure 4 (a) is an occlusion scene at an intersection, where the road conditions are relatively complex. It can be clearly seen that the turning and straight-moving vehicles in front cause visual occlusion during driving, and the shadows of the trees on both sides of the road also form an occlusion phenomenon with the vehicles, making the horizontal outline of the target vehicle unclear. The intuitive manifestation of the detection result is that the higher the degree of occlusion of the target, the greater the positioning uncertainty. As the number of training iterations increases, the uncertainty of the bounding box gradually decreases, the model's cognitive ability improves, and it can detect previously missed targets. The final detection result is basically consistent with the true value label.

[0141] Small target scene

[0142] Figure 4 (b) The scene contains smaller targets. The target vehicles that are closer to the main vehicle in the straight line of view have clear outlines, and the uncertainty of bounding box regression is small; while the target vehicles in the distance are too far away from the main vehicle, which has caused a certain visual blur effect, so the uncertainty is larger. For some pedestrians and riders on the left side of the image, they are small in themselves and are blocked by trees around them. The model's cognitive results are very uncertain. Compared with the true value label, there is still a small and blocked target missing in the final detection result.

[0143] Dark scenes

[0144] Figure 4 (c) is a dark experimental scene. The target vehicle in front of the main vehicle on the corresponding lane has a clear outline and is relatively close, so the detection results are relatively certain. In contrast, the other vehicles parked on the right sidewalk have high uncertainty due to the overall dim light and the obstruction of trees. In the middle training iteration, it can be seen that there is a false detection on the right. Observing the surrounding environment of the false detection result, it is likely that other obstacles on the sidewalk were redundantly detected. Finally, after multiple iterations using uncertainty, a result similar to the true value label was obtained.

[0145] In summary, when there are occlusions, small targets, and dimly lit scene samples, the model will give higher uncertainty. This uncertainty has a positive impact on the credibility of the detection results given by the model. Uncertainty quantification and fusion in iterative training ensures the generalization performance of the SSOD algorithm in complex and diverse autonomous driving scenarios.

[0146] Compared with the prior art, the present invention has the following beneficial effects:

[0147] The present invention adopts an uncertainty-based semi-supervised target detection framework suitable for the field of autonomous driving, and addresses the problem of reduced algorithm detection performance caused by lack of labels by introducing an uncertainty mechanism into the semi-supervised target detection task.

[0148] Aiming at the characteristics of semi-supervised object detection tasks, a general SSOD algorithm framework based on uncertainty is proposed. As far as we know, this is the first time that a training strategy based on two uncertainties is proposed for semi-supervised object detection tasks.

[0149] Using the uncertain semi-supervised target detection model training framework, an uncertainty fusion mechanism of pseudo labels and the Fuzzy-NMS algorithm are proposed to fuse the fuzzy pseudo labels generated by multiple iterative reasoning to improve their quality and optimize the training of the student model.

[0150] At the data enhancement level, elliptical rotation is proposed and combined with random replication. By enhancing the unlabeled data separately and fully combining the fuzzy pseudo-label information generated by reasoning, the model can obtain more stable and accurate output.

[0151] The above is a detailed introduction to the uncertainty-aware semi-supervised target detection method in an autonomous driving environment provided by the present application. The description of the specific embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. An uncertainty-aware semi-supervised target detection method in an autonomous driving environment, characterized in that: Includes steps: S1. Supervised learning of labeled data, using data augmentation and uncertainty loss function to obtain a teacher model with the ability to predict uncertainty; S2, the teacher model uses unlabeled data reasoning to generate fuzzy pseudo labels; S3. Using fuzzy pseudo-labels as the learning objective of the student model, we quantify the similarity of the uncertainty outputs of the teacher and student models.

2. The uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to claim 1, characterized in that: Uncertainty is introduced in model training, model parameters are regarded as parameter distribution, model likelihood is converted into Gaussian likelihood, and each output of the model is independently generated by the same probability distribution, as shown in the following formula: where {x i ,y i } represents the labeled samples input during supervision, and {f w (x),σ} represents the uncertain results output by the teacher model; The bounding box regression result obtained by the detection network is expressed as [c x ,c y ,w,h]∈R 4 , mainly predicting the center coordinates of the bounding box and the size of the overall box, and encoding each regression result accordingly; Model the uncertainty in the above regression results: Assume that each parameter in the regression result follows a Gaussian distribution and the variance of the prediction result is σ 2 , according to the maximum likelihood estimation theory, the parameters of the detection model are optimized as shown in the following formula: Where ω represents the neural network model parameters, ω* represents the optimized parameters, using the maximum conditional probability (argmax), y represents the output variable, x represents the input variable, p(y|x,w) represents the probability density function, First, we discretize it into a continuous multiplication form, then take the natural logarithm to process the probability density function log(p(y|x,w)) in the product form, and finally use the probability density function form of Gaussian distribution, where f(x) represents the mean predicted by the model, y i is the true value of the sample, σ i 2 is the variance.

3. The uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to claim 2, characterized in that: The uncertainty loss function for teacher model training is designed as shown in the following formula: Uncertainty loss function L v l The actual value y is calculated i and the predicted value f w (x i ) and is weighted by exponential decay exp(-α i ) adjusts the influence of each sample, where α i represents the uncertainty index; Uncertainty loss function For the four bounding box coordinate parameters [Δc x ,Δc y ,Δw,Δh] for regression fitting, which includes two parts: one is the residual regression term obtained by using random samples through the model, and the other is the regularization term using the uncertainty of the parameters.

4. The uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to claim 3, characterized in that: The detailed steps of the teacher model using unlabeled data reasoning to generate fuzzy pseudo labels include: S201, Pseudo-label and self-training: Pseudo-label learning introduces a method of training deep neural networks in a semi-supervised manner. It enhances the learning process by generating proxy labels on unlabeled data, and directly infers the generation using prediction functions or variants of pseudo-labels without any supervised training. It uses proxy labels together with labeled data as training targets to mine additional useful information, and supplements and improves the semi-supervised model training process. S202, define fuzzy pseudo-labels: the teacher model directly infers the unlabeled data input to obtain fuzzy pseudo-labels, and the obtained fuzzy pseudo-labels also have uncertainty; the uncertainty regression of the fuzzy pseudo-labels is expressed as: p(t * |x)={t * ,α * }; where t * are the bounding box coordinates of the blurred pseudo-label, α * It is about * Homologous uncertainty of specific parameters; S203, uncertainty fusion mechanism: fuse the predicted regression uncertainty and positioning information separately.

5. The uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to claim 4, characterized in that: In the semi-supervised object detection task, a teacher-student model is used for self-training, which includes two stages: the student learns from the teacher and the teacher learns from the student's verification loss. The model uses the pseudo labels inferred by the teacher model and its own true labels to improve the learning quality of the student model, and updates the teacher model by evaluating the policy gradient calculated by the student model on the retained validation set.

6. The uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to claim 5, characterized in that: Detailed steps to fuse the predicted regression uncertainty and positioning information separately: Sort the bounding box results from high to low confidence, compare these boxes with the box with the highest confidence, and calculate the IoU value; Determine whether the IoU of the two detection boxes is greater than the threshold. If it is greater than the threshold, proceed to the next step; The IoU calculation and comparison process is repeated until the detection box with the lowest confidence score is operated on.

7. The uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to claim 6, characterized in that: The uncertainty fusion mechanism also includes the fusion update of fuzzy pseudo labels in different iterative training stages; The uncertainty fusion mechanism performs uncertainty fusion on the fuzzy pseudo-labels generated by the previous and subsequent iterations to obtain a more stable result, and repeats this updating process until the training is completely completed.

8. The uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to claim 7, characterized in that: The similarity calculation formula for quantifying the uncertainty output of the teacher and student models is: It is an overall description of the loss function of the teacher model and the student model in the unsupervised process; The first part is the summation symbol ∑i=1N l , it is from i = 1 to N l For each value i, the function L is calculated l (f t (x i l ,v t ,w t )) and add these values ​​together, L l Represents the loss function of the teacher model for supervised learning to generate pseudo labels, using the cross entropy loss function; where N l represents the total number of labeled samples; f t represents the teacher model; x i l represents the i-th input sample of supervised learning; v t represents the uncertainty of the teacher model; w t represents the teacher model weight; Part II u is the total number of unlabeled image samples, L u represents the unsupervised loss of unlabeled data, and also uses the packaged cross entropy loss function. * represents the pseudo label inferred by the teacher model, γ is the weight coefficient between supervised and unsupervised losses, u on x represents unlabeled input, and v s represents the uncertainty of the student model; w s represents the student model weight; The uncertainty of the fuzzy pseudo-label data and the uncertainty of the student model are combined, and based on the theory of KL divergence, the similarity of the output distribution of the two models is quantified as the optimization goal for secondary training: L KLD represents the Kullback-Leibler divergence, an asymmetric measure of the difference between two probability distributions, where p t* (y|x) represents the probability density function of the teacher model for y given the input x, p s (y|x) represents the probability density function of the student model for y given the input x; logp t* (y|x) / p s (y|x) is the logarithmic ratio of the two probabilities. It mainly compares the outputs of the two probability functions, and then integrates the natural logarithm of the teacher model probability multiplied by its difference from the student model distribution to obtain the KL divergence. A KL divergence of zero indicates that the two distributions are exactly the same; a non-zero value indicates that there is a difference between them, and the larger the value, the more significant the difference; The corresponding uncertainty loss function in the unsupervised process is designed as follows: in, It is a relative uncertainty representation in unsupervised training, combining the two variances of the output as the final description of uncertainty; L v u Indicates that the uncertainty similarity between the teacher model and the student model is included in the loss function for evaluation. s，j represents the uncertainty of the j-th output of the teacher model; σ t*，j represents the uncertainty of the j-th output of the student model; the former is multiplied by the latter to the power of -1 to obtain the relative uncertainty representation; f t*，j (x) represents the output of the teacher model after unsupervised learning under the input x; f s，j (x) represents the output of the student model under the input x, and the absolute value of the two is calculated.

9. The uncertainty-aware semi-supervised target detection method in an autonomous driving environment according to claim 8, characterized in that: The entire uncertainty-aware semi-supervised target detection method uses two data enhancement methods: ellipse rotation and random replication. Use an inscribed ellipse instead of a circumscribed rectangle to fit the shape of the object; After rotating the image, the circumscribed rectangle of the largest inscribed ellipse of the rotated bounding box is taken as the enhanced label. The largest inscribed ellipse is based on the bounding box positioning information [c x ,c y ,w,h]∈R 4 Generate, the specific calculation equation is: The above formula is one of the standard equations of an ellipse, which defines an ellipse centered at point (cx, cy) with a horizontal semi-axis length of 2w and a vertical semi-axis length of 2h. Its maximum inscribed ellipse is calculated using the bounding box positioning information [cx, cy, w, h]. The IoU threshold is dynamically optimized and updated by randomly rotating the angle to meet the threshold requirement, and the next step of fusion and loss calculation is performed according to the scheme described in the uncertainty fusion mechanism of fuzzy pseudo-labels; The uncertainty threshold update formula of IoU is: Thr(θ) represents the IoU uncertainty threshold obtained when the input is θ, θ0 is the "initial rotation angle parameter corresponding to the lowest IoU threshold of 0.5", when θ=θ0, Thr(θ)=0.5; on 0-π / 2, θ is less than θ0, and when it is greater than 0, θ becomes smaller, sin2θ becomes smaller, and the denominator is a constant, so the square term becomes smaller as a whole, and Thr(θ) becomes larger; Where θ is the rotation angle, and θ0 is the initial rotation angle parameter corresponding to the lowest IoU threshold of 0.5.

Images