An object detection active learning method based on double pseudo-label regression entropy

Abstract

The application relates to a double pseudo-label regression entropy target detection active learning method, which can solve the inaccuracy problem caused by the particularity of the detection task and the indirect evaluation of the information amount of the unlabeled sample. The method comprises the following steps: constructing a double pseudo-loss, backtracking the initial prediction box to the candidate box, enhancing the candidate box to obtain an enhanced prediction box, and constructing the vector difference form of the two; calculating the regression entropy based on the double pseudo regression loss, predicting the candidate box after N times of Gaussian enhancement, processing the expectation and entropy value of the N vector differences, and fusing them; calculating the pseudo cross-entropy loss as the classification information amount; fusing the regression information amount and the classification information amount; traversing and summing the multiple initial prediction boxes obtained by prediction to the multiple candidate boxes in the ROI stage to obtain the information amount of the final unlabeled sample; the method can realize more accurate detection sample information amount evaluation, and opens up a new idea for the active learning method based on target detection.

Description

Technical Field

[0001] This invention relates to the field of active learning, and in particular to an active learning method for target detection based on double pseudo-label regression entropy. Background Technology

[0002] Training object detection models using deep learning techniques requires numerous labeled samples, and obtaining these labeled samples is a costly endeavor. With the advent of active learning, the cost of sample labeling has been significantly reduced. By evaluating the information content of unlabeled samples and selecting those with high information content for annotation, the model can achieve performance comparable to training with all samples.

[0003] Various active learning methods have been integrated into object detection tasks. Most of these methods indirectly evaluate the uncertainty, feature distribution, and gradient of unlabeled detection samples, calculating the information content of these samples based on the active learning paradigm of classification tasks, thereby selecting high-information samples for annotation. However, indirect evaluation of information content is not precise enough and cannot provide comprehensive information increments for the training process. Furthermore, due to the specific nature of object detection tasks, it is necessary to consider not only classification information content but also regression information content. Since there are one or more instances in the unlabeled detection samples, directly evaluating the entire unlabeled detection sample cannot accurately represent its information content. Summary of the Invention

[0004] The purpose of this invention is to avoid the inaccuracies caused by the specific nature of the detection task and the indirect evaluation of the information content of unlabeled detection samples. It proposes an active learning method for object detection based on double pseudo-label regression entropy. The information content of unlabeled detection samples can be directly evaluated through loss. A designed double pseudo-loss label is used to evaluate the regression information content of unlabeled detection samples. A regression information content evaluation module and a classification information content evaluation module are constructed, and certain candidate box selection principles are designed, thereby enabling a more accurate evaluation of unlabeled detection samples. The technical solution is as follows:

[0005] An active learning method for object detection based on double pseudo-label regression entropy, the steps of which are as follows:

[0006] Step 1: Construct a double pseudo-regression loss;

[0007] (1) Select the Faster R-CNN object detection framework for model training;

[0008] (2) Feed the unlabeled detection samples into the trained model for testing, and obtain the prediction score and the initial prediction box t. i,j,w,h ;

[0009] (3) The initial prediction box t i,j,w,hPerform backtracking to find the candidate bounding box t' corresponding to the ROI stage. i,j,w,,h For candidate box t' i,j,w,,h After Gaussian enhancement, the enhanced prediction box t1 is obtained. i,j,w,h The backtracking operation is performed using the keep variable within the Faster R-CNN framework.

[0010] (4) The initial prediction box t i,j,w,h With enhanced prediction box t1 i,j,w,h Construct into vector form and Perform vector interpolation to obtain the double pseudo-regression loss Pseudo_Reg. loss The double pseudo-regression loss characterizes the difference between the predicted box and the ground truth box from three aspects: the magnitude of the offset norm, the angle of the offset, and the degree of difference in the offset norm.

[0011]

[0012] Step 2: Obtain the regression entropy of the unlabeled detection samples based on the double pseudo-regression loss;

[0013] (1) Initial prediction box t for unlabeled detection samples i,j,w,h Backtracking to the candidate box t' at the ROI stage i,j,w,,h For candidate box t' i,j,w,,h Perform N Gaussian enhancements to obtain the enhanced prediction box t1. i,j,w,h ..., Enhanced prediction box tn i,j,w,h ;

[0014] (2) Using the double pseudo-regression loss calculation method from the first step, each enhanced prediction box tn obtained after Gaussian enhancement is... i,j,w,h With the initial prediction box t i,j,w,h Perform vector interpolation to obtain a list of vector interpolations (Pseudo_Reg). loss 1,Pseudo_Reg loss 2,...,Pseudo_Reg loss n);

[0015] (3) (Pseudo_Reg) loss 1,Pseudo_Reg loss 2,...,Pseudo_Reg loss The loss values ​​(loss1, loss2, ..., loss) are then processed using softmax to obtain the normalized loss values. n );

[0016] (4) For (loss1, loss2, ..., loss nEntropy calculation is performed to obtain the regression entropy of the unlabeled test samples;

[0017]

[0018] Step 3: Evaluate the amount of regression information in unlabeled detection samples by using the expected value and entropy value of the regression loss under different enhancement scales;

[0019] (1) The normalized loss values ​​(loss1, loss2, ..., loss) obtained in the second step n ) Calculate the expected value;

[0020] (2) The regression entropy and expected value of the pseudo-regression loss under different Gaussian enhancement scales are fused to evaluate the regression information content of unlabeled detection samples;

[0021]

[0022] Step 4: Assess the amount of classification information in the unlabeled test samples;

[0023] Find the initial prediction box t i,j,w,h The category label corresponding to the highest confidence level is used as the pseudo-label p. i The pseudo-classification loss Pseudo_Cla is obtained. loss As a categorized information quantity;

[0024] Information Cla =-log(p i )

[0025] Step 5: Regression Information Reg Information related to classification Cla To merge;

[0026] Information = Information Reg +Information Cla

[0027] Step 6: Selection of anchor frames in unlabeled test samples;

[0028] Multiple initial prediction boxes obtained from the prediction Backtracking to the ROI stage, multiple candidate boxes are identified. The information content of each candidate box is evaluated using the methods from the previous five steps. The information content of multiple candidate boxes is summed to obtain the final information content of the unlabeled detection sample.

[0029]

[0030] Step 7: Active Learning Paradigm

[0031] Randomly select base class data for initial model training. Calculate the information content of unlabeled detection samples using the above steps, then select high-information samples for manual labeling to train the model. Iterate this process repeatedly until the model achieves good accuracy or the budget is exhausted.

[0032] The beneficial effects of this invention are:

[0033] The proposed active learning method for object detection based on double pseudo-label regression entropy not only effectively establishes double pseudo-labels in the regression part for evaluating regression loss, but also provides a more accurate assessment of the information content in the regression part. The selection principles for candidate boxes and the integration of classification information help construct an information content that better reflects the entire image. In summary, this method opens up a new avenue for active learning methods based on object detection. Attached Figure Description

[0034] Figure 1 This is a diagram illustrating the overall framework of active learning for target detection in this invention.

[0035] Figure 2 This is the vector difference form that characterizes the initial prediction box and the enhanced prediction box constructed in this invention;

[0036] Figure 3 In this invention, the offset norm is used to characterize the difference between the predicted bounding box and the ground truth bounding box.

[0037] Figure 4 In this invention, the angle of the offset is used to characterize the difference between the predicted box and the truth box;

[0038] Figure 5 In this invention, the degree of difference in the offset norm is used to characterize the difference between the predicted box and the ground truth box; Detailed Implementation

[0039] To make the technical solution of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings. The present invention is implemented in specific steps:

[0040] Step 1: Construct a double pseudo-regression loss;

[0041] (1) Select the Faster R-CNN object detection framework for model training;

[0042] MobileNetV2 was selected as the pre-trained model, and the Faster R-CNN two-stage object detection framework was used for training to obtain the trained model.

[0043] (2) Feed the unlabeled detection sample un_sample into the trained model Model for testing, and obtain the prediction score and the initial prediction box t. i,j,w,h As shown in formula (1);

[0044] t i,j,w,h ,score=Model(un_sample) Formula (1)

[0045] Among them, t i,j,w,h In this context, i, j, w, and h represent the proportions of horizontal translation, vertical translation, width scaling, and height scaling, respectively, and score represents the probability of being predicted as each category.

[0046] (3) The initial prediction box t i,j,w,h Perform backtracking to find the candidate bounding box t' corresponding to the ROI stage. i,j,w,,h For candidate box t' i,j,w,,h After Gaussian augmentation, the original model (Model) is used for testing to obtain the augmented prediction box t1. i,j,w,h The backtracking operation is performed using the keep variable within the Faster R-CNN framework, which includes processes such as removing anchor boxes with confidence levels below the set threshold and NMS suppression.

[0047] (4) The formula for calculating the regression loss function in target detection is shown in formula (2);

[0048]

[0049] The initial prediction box t i,j,w,h With enhanced prediction box t1 i,j,w,h Construct into vector form and Then, vector interpolation is performed to obtain the double pseudo-regression loss Pseudo_Reg. loss As shown in formula (3);

[0050]

[0051] The double pseudo-regression loss characterizes the difference between the predicted box and the ground truth box from three aspects: the magnitude of the offset norm, the angle of the offset, and the degree of difference in the offset norm.

[0052] Step 2: Obtain the regression entropy of the unlabeled detection samples based on the double pseudo-regression loss calculated in Step 1;

[0053] (1) The initial prediction box t of the unlabeled test samples i,j,w,h Backtracking to the candidate box t' at the ROI stage i,j,w,,h For candidate box t' i,j,w,,h Perform N Gaussian enhancements to obtain the enhanced prediction box t1.i,j,w,h ..., Enhanced prediction box tn i,j,w,h Experiments showed that the optimal effect was obtained when N was set to 5.

[0054] (2) Using the double pseudo-regression loss calculation method from the first step, the enhanced prediction box tn obtained after each Gaussian enhancement is... i,j,w,h With the initial prediction box t i,j,w,h Construct a vector form and then perform vector difference operations, as shown in formula (4), to obtain the vector difference list (Pseudo_Reg). loss 1,Pseudo_Reg loss 2,...,Pseudo_Reg loss n);

[0055]

[0056] (3) The vector difference list (Pseudo_Reg) loss 1,Pseudo_Reg loss 2,...,Pseudo_Reg loss n) is processed using softma x, as shown in formula (5), to obtain the normalized loss values ​​(loss1, loss2, ..., loss n );

[0057]

[0058] (4) Next, the normalized loss values ​​(loss1, loss2, ..., loss) are processed. n The entropy value is calculated to obtain the regression entropy of the unlabeled detection sample, as shown in formula (6).

[0059]

[0060] Step 3: Evaluate the amount of regression information in the unlabeled test samples;

[0061] (1) The normalized loss values ​​(loss1, loss2, ..., loss) obtained in the second step n The expected value is obtained as shown in formula (7).

[0062]

[0063] (2) The regression entropy and expected value of the pseudo-regression loss under different Gaussian enhancement scales are fused to evaluate the regression information of the unlabeled detection samples, as shown in formula (8).

[0064]

[0065] Step 4: Assess the amount of classification information in the unlabeled test samples;

[0066] (1) Cross-entropy loss is used as the classification loss function in object detection. The calculation formula is shown in formula (9).

[0067] Cla loss =-∑y i ·log(p i Equation (9)

[0068] Among them, y i To represent the actual label category, p i This represents the label category predicted by the trained model. When y i When it does not belong to the true category label, y i Take 0; when y i When it belongs to the true category label, y i Let's take 1. Therefore, formula (9) can be simplified to formula (10).

[0069] Cla loss =-log(p i Equation (10)

[0070] (2) For the initial prediction box t i,j,w,h The predicted score is used to find the category label with the highest confidence level as the pseudo-label p. i Based on this, the pseudo-classification loss Pseudo_Cla is obtained. loss As the classification information, it is shown in formula (11).

[0071] Information Cla =-log(p i Equation (11)

[0072] Step 5: Fusion of regression and classification information of unlabeled test samples;

[0073] Regression Information Reg Information related to classification Cla The fusion is performed as shown in formula (12).

[0074] Information = Information Reg +Information Cla Equation (12)

[0075] Step 6: Selection of anchor frames in unlabeled test samples;

[0076] (1) Due to the influence of multiple instances and model detection uncertainty, multiple initial prediction boxes will be generated when unlabeled detection samples are fed into the trained model for testing. Multiple initial prediction boxes Backtrack to multiple candidate boxes at the corresponding ROI stage.

[0077] (2) Traverse multiple candidate boxes, evaluate the information content of each candidate box using the method of the first five steps, sum the information content of multiple candidate boxes, and obtain the information content of the final unlabeled detection sample, as shown in formula (13).

[0078]

[0079] Where M is the number of candidate boxes, (Information Reg +Information Cla ) m This represents the information content of the m-th candidate box.

[0080] Step 7: Active Learning Paradigm;

[0081] Based on the unlabeled training set of VOC2012, 5% of the base class data was randomly selected and labeled for initial model training. The information content of the remaining unlabeled detection samples was evaluated using the above steps, and the information content values ​​were sorted from highest to lowest. The top 10% of samples with high information content were selected, labeled, and used in conjunction with the base class data for model training. This process was iterated repeatedly until the model achieved good accuracy or the budget was exhausted.

Claims

1.A target detection active learning method based on double pseudo-label regression entropy, characterized in that, The steps are as follows: S1: Construct a double pseudo-regression loss module, including: (1) First, the image object detection model is trained using the Faster R-CNN framework; (2) Input the unlabeled detection image to be evaluated into the trained Faster R-CNN and obtain its prediction confidence score. and the initial predicted bounding box subscript These represent the proportions of horizontal translation, vertical translation, width scaling, and height scaling, respectively. (3) using the keep variable in the Faster R-CNN model to perform a regression mapping operation on the initial prediction box to locate to the corresponding ROI candidate box ; applying Gaussian noise enhancement to the candidate box, and obtaining an enhanced prediction box after re-prediction by the model ; (4) the initial prediction box and the enhanced prediction box are represented as vector forms and , the vector difference of the two is calculated to obtain the double pseudo regression loss ; this loss characterizes the difference between the prediction box and the pseudo ground truth box from three dimensions of the norm size of the offset vector, the offset direction angle, and the difference of the offset amount: S2: Based on the double pseudo-regression loss module, perform regression entropy operation on the unlabeled detection image, including: (1) For the ROI candidate frame obtained in step one Apply 5 independent Gaussian enhancement operations to obtain an enhanced prediction frame sequence ; (2) Using the double pseudo regression loss calculation method in step one, the prediction box obtained after each Gaussian enhancement with the initial prediction frame applying vector difference operation to obtain regression loss sequence ; (3) For the regression loss sequence Perform softmax normalization to obtain the normalized loss distribution. : (4) to the normalized loss distribution Applying an entropy value calculation to obtain a regression entropy of the unlabeled detection image : The regression entropy value quantifies the uncertainty of the model's regression prediction under different enhancement conditions; S3: Based on the regression entropy evaluation results, calculate the regression information content, including: (1) For the normalized loss distribution obtained in step two the expected value is calculated : (2) The regression entropy and the expected value of the pseudo regression loss distribution under different Gaussian enhancement scales are fused to comprehensively evaluate the regression information of the unlabeled detection image : S4: Evaluate the classification information content of unlabeled detection images the highest confidence of the initial prediction box corresponding to the class label as a pseudo label , get the pseudo classification loss as its classification information quantity; S5: fusing the regression information volume with the classification information volume to perform a fusion operation; S6: Selection of anchor frames in unlabeled detection images Multiple initial prediction boxes obtained from the prediction Backtracking to multiple candidate boxes in the ROI stage, traversing multiple candidate boxes, evaluating the information content of each candidate box using the method of the first five steps, summing the information content of multiple candidate boxes, and obtaining the information content of the final unlabeled detection image. wherein M is the number of candidate boxes, represents the information amount of the th candidate box. S7: Active Learning Paradigm Randomly select base class images for initial model training; calculate the information content of unlabeled detection images through the above steps, and then select high information content images for manual labeling for model training. Repeat this process until the model achieves good accuracy or the budget is exhausted.

Images