A pseudo-label-based target detection training method

Abstract

The present application relates to a kind of target detection training methods based on pseudo label, belong to semi-supervised learning field.The classification output of the present application is processed in multiple stages, and the class output probability greater than or equal to the first threshold value is as the class pseudo label of this target, the class output probability value greater than the second threshold value and less than the first threshold value is marked as special class, and the class output probability value less than or equal to the second threshold value is as background class;In optimization loss function, special class has no contribution to loss function;The output of target detection algorithm model includes classification branch, regression branch and regression frame reliable judgment branch, on the basis of the output of regression branch, the IoU value of the output result of regression branch and the labeled frame is calculated, and the IoU value is used as the learning goal of regression frame reliable judgment branch, and the corresponding regression branch output is more accurate for the output value of regression frame reliable judgment branch higher.The present application maintains the diversity of the output of new model, maintains the class accuracy of pseudo label, and also can guarantee the position accuracy of pseudo label.

Description

Technical Field

[0001] This invention belongs to the field of semi-supervised learning, specifically relating to a target detection training method based on pseudo-labels. Background Technology

[0002] The explosive growth of deep learning technology benefits from two fundamental supports: computing power and data resources. Data resources refer to large amounts of labeled datasets, such as the Mscoco dataset for object detection and segmentation, which contains over 200,000 labeled images, 1.5 million object instances, and 80 object categories. Large-scale annotation is extremely labor-intensive and resource-intensive; therefore, reducing the amount of labeled data required for deep learning algorithms has become a very real research need.

[0003] Semi-supervised learning strategies are a technological approach to address these needs. They leverage large amounts of unlabeled data from labeled datasets to enhance the expressiveness and generalization ability of algorithmic models. The pseudo-label algorithm involved in this invention is a semi-supervised learning method derived from the idea of ​​entropy regularization. It is simple and effective to implement and easily integrated with existing classification and detection algorithms.

[0004] Entropy regularization is also a form of entropy minimization. Its principle is to encourage the accuracy of the model's probability predictions. It is believed that the more accurate the model's predictions for new data (the higher or lower the probability), the more reliable its predictions are.

[0005] The typical process of a pseudo-labeling algorithm involves first training a teacher model on a base labeled dataset. This model is then applied to unlabeled data, generating predictions (candidate pseudo-labels). These predictions are then filtered to obtain more reliable predictions. These predictions and the data themselves constitute a dataset, which is called a pseudo-label dataset to distinguish it from the similar manually labeled base training dataset. This pseudo-label dataset is then combined with the base dataset to form a new training dataset, which is then used to retrain the student model. This allows the model to transfer and learn information from the pseudo-label data, thereby improving its generalization ability.

[0006] The key here lies in obtaining reliable pseudo-labels. From a task type perspective, object detection tasks include not only object classification but also object location prediction. For category prediction, pseudo-label object detection algorithms typically take the category corresponding to the result with the highest category confidence or greater than a certain threshold (e.g., 0.9) in the detection model as the pseudo-label for the object's category. For location, either the target bounding box is directly extracted based on the category confidence as the pseudo-label, or different detection boxes of the same object are obtained through data augmentation transformation, and these boxes are then combined with category confidence as weights to generate the final target bounding box. The statistically augmented result is then used as the pseudo-label for the location.

[0007] The above approach has two problems. First, in terms of classification, the pseudo-label algorithm selects pseudo-labels with high confidence levels, and detection results with confidence levels below this threshold are treated as negative or background. This simply reinforces the model's output of high-confidence results on unlabeled data, suppresses the potential output of the base model, and limits the ability to extract information from unlabeled data and the diversity of results. Based on the aforementioned entropy regularization idea, it is certain whether the confidence level is high or low, but it is uncertain for intermediate confidence levels.

[0008] Secondly, regarding the localization of the target bounding box, the target bounding boxes extracted by the aforementioned pseudo-label strategy are treated as ground truth during subsequent training. This means that the regression error objective function for location encourages the corresponding output to align with the ground truth. Therefore, the accuracy of the ground truth location is crucial. However, in object detection network models, the classification branch for category prediction and the location regression branch in the output of the final network layer are two independent branches. This means that a higher confidence level in the classification branch does not necessarily guarantee a more accurate location in the corresponding location regression branch. In contrast, the aforementioned pseudo-label strategy determines the accuracy of location prediction based on the confidence level of the category prediction. Summary of the Invention

[0009] (a) Technical problems to be solved

[0010] The technical problem this invention aims to solve is how to provide a target detection training method based on pseudo-labels. This addresses the issues in pseudo-label algorithms where high-confidence pseudo-labels are generated, and detection results below a certain confidence threshold are treated as negative or background classes. This approach merely reinforces the model's output of high-confidence results on unlabeled data, suppresses the potential output of the base model, and limits the ability to extract information from unlabeled data and the diversity of results. Furthermore, the pseudo-label strategy determines the accuracy of location prediction based on the confidence level of class prediction.

[0011] (II) Technical Solution

[0012] To address the aforementioned technical issues, this invention proposes a target detection training method based on pseudo-labels. The method performs multi-level processing on the classification output, generally divided into three levels: categories with output probabilities greater than or equal to a first threshold are designated as pseudo-labels; categories with output probabilities greater than a second threshold but less than the first threshold are labeled as special classes; and categories with output probabilities less than or equal to the second threshold are designated as background classes. In optimizing the loss function, special classes do not contribute to the loss function, thus preserving the model's ability to output to these uncertain targets.

[0013] The output of the object detection algorithm model includes a classification branch, a regression branch, and a reliable judgment branch for the regression boxes. Based on the output of the regression branch, the IoU value between the regression branch output and the labeled box is calculated, and this IoU value is used as the learning target of the reliable judgment branch for the regression boxes. The higher the output value of the reliable judgment branch for the regression boxes, the more accurate the output of the corresponding regression branch.

[0014] Furthermore, the target detection algorithm model includes a backbone network and an RCNN Head. The input image is processed by the backbone network to extract features, then passes through the RoIAlign operation, and enters the RCNN Head. After passing through convolution and two fully connected layers, it outputs a classification branch for the target category, a regression branch for the target location, and a reliable judgment branch for the regression box. The reliable judgment branch for the regression box is a box score output branch that has the accuracy to predict the target location.

[0015] Furthermore, the first threshold is 0.8.

[0016] Furthermore, the second threshold is 0.5.

[0017] Furthermore, the method includes a basic dataset training component: the labeled basic training data sample set is referred to as D. base The corresponding sample image is im base_i 'i' is the sample index, and the corresponding manually labeled information is 'label'. base_i =(Gcls0,Gbox0,...Gcls) j Gbox j ...Gcls N Gbox N ), where j is the index of the labeled instance corresponding to the i-th sample, and Gcls j For the category of the j-th labeled instance, Gbox j The position of the bounding box for the j-th labeled instance;

[0018] Sample image im base_i and corresponding labeled data base_i To create training sample data pairs, first, im base_i Input the model, and obtain the predicted output of the three branches of the model as Pboxes. k Pcls k and Pscore k k represents the index of the predicted instance result of the model for image i. The annotation information and output information are then processed through a Pbox. k and Gbox j The Intersection over Union (IoU) is used for matching. If the IoU is greater than a third threshold, a matching pair is constructed to determine the positive sample. Let p be the value of the positive sample. k =1, and the remaining predicted outputs are treated as negative samples, denoted as p.k =0; For a matching positive sample, the pair of Pboxes is set to 0. k and Gbox j The cross-union ratio (CUIR) is used as the ground truth corresponding to the box score at that time, i.e., the Pscore. k The regression target value is denoted as GPiou. kj Its model loss function is

[0019] L=E(Pcls k ,Gcls j )+Reg(Pbox k Gbox j )*p k +Reg(Pscore k ,GPiou kj )*p k Here, E represents the classification loss function, Reg represents the regression loss function, and p... k The regression loss is calculated only for positive samples; thus, a model model_base is obtained by training on the base dataset.

[0020] Furthermore, the third threshold is 0.5.

[0021] Furthermore, the stochastic gradient descent optimization method was used during training on the base dataset, with the learning rate set to 0.001.

[0022] Furthermore, the method also includes a pseudo-label generation part: representing unlabeled data as Imnovel n Where n is the ordinal number of the unlabeled image, the model model_base is used to predict its value, and the prediction result is denoted as PNcls. m PNbox m PNscore m , m represents the index of the model prediction result instance for the unlabeled image n, taken as PNcls m *PNscore m Detection results greater than or equal to the first threshold are used as pseudo-labels, and these pseudo-labels are denoted as PGNcls. m PGNbox m PGNscore m For PNcls m *PNscore m Prediction results less than or equal to the second threshold are not recorded and are treated as background classes; the remaining prediction outputs are treated as intermediate results, and pseudo-labels are output normally. These pseudo-labels are specially marked for subsequent loss function calculations. The results of these reliable labels and intermediate result labels constitute the pseudo-label sample dataset.

[0023] Furthermore, the method also includes a retraining part: First, the basic sample dataset and the pseudo-label sample dataset are fused and reconstructed. The basic sample dataset is randomly selected, and its quantity is twice the size of the aforementioned pseudo-label sample dataset. This sample is then mixed with the aforementioned pseudo-label sample dataset to form a retraining sample dataset. Then, the model is trained based on the retraining sample dataset. When calculating the loss function, for the intermediate result instances labeled with pseudo-labels, the corresponding loss errors are all set to zero in the loss function, meaning that these instances do not contribute to the update of the model parameters.

[0024] Furthermore, the best model parameters on the retrained sample dataset, based on the general test metric mAp50 for object detection, are used as the new model parameters.

[0025] (III) Beneficial Effects

[0026] This invention proposes a target detection training method based on pseudo-labels. The pseudo-label algorithm of this invention adds multi-level threshold label processing to inherit the output capability of the base model on the intermediate results on the new dataset, thereby maintaining the output diversity of the new model. This invention adds a regression position quality judgment branch and combines it with the category probability output to filter and extract pseudo-labels, which can ensure the position accuracy of pseudo-labels while maintaining the category accuracy of pseudo-labels. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the model framework of the present invention. Detailed Implementation

[0028] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0029] To address the aforementioned problems of entropy regularization-based pseudo-label algorithms in target detection applications, this invention proposes the following improvements to obtain reliable and effective pseudo-labels:

[0030] The classification output is processed in multiple levels, generally divided into three levels: class output probabilities greater than or equal to the first threshold (e.g., greater than or equal to 0.8) are used as pseudo-labels for the target class; class output probabilities greater than the second threshold but less than the first threshold (e.g., greater than 0.5 and less than 0.8) are marked as special classes; and class output probabilities less than or equal to the second threshold (e.g., less than or equal to 0.5) are used as background classes. In optimizing the loss function, special classes do not contribute to the loss function, that is, the model's ability to output for these uncertain targets is preserved.

[0031] To address the issue of accurate bounding box regression, this invention adds a reliable bounding box judgment branch to the existing classification and regression branches of the object detection algorithm model. Borrowing from the Mask Scoring R-CNN approach, the IoU value (between 0 and 1, with higher values ​​closer to the true value) between the regression branch output and the labeled bounding box is calculated based on the regression branch output. This value is then used as the learning target for the reliable bounding box judgment branch. Higher output values ​​from the reliable bounding box judgment branch correspond to more accurate regression branch outputs, thus transforming the regression problem into a classification problem.

[0032] like Figure 1 As shown, the overall structure of the present invention includes the following parts:

[0033] 1) Model structure:

[0034] Figure 1 This is based on a typical object detection network architecture. The object detection algorithm model of this invention includes a backbone network and an RCNN Head. The input image is processed by the backbone network to extract features, then passes through RoIAlign (a classic object detection operation, no explanation needed), and enters the RCNN Head. After convolution and two fully connected layers, it outputs a classification branch for the object category and a regression branch for the object location. The difference from the classic model is that the network model output part adds a reliable box judgment branch, which is a box score output branch that determines the accuracy of the object location prediction.

[0035] The present invention also includes a basic dataset training part, a pseudo-label generation part, and a retraining part;

[0036] 2) Training part of the basic dataset:

[0037] Here, the labeled basic training data sample set is referred to as D. base The corresponding sample image is im base_i 'i' is the sample index, and the corresponding manually labeled information (ground truth) is the label. base_i =(Gcls0,Gbox0,...Gcls) j Gbox j ...Gcls N Gbox N ), where j is the index of the labeled instance corresponding to the i-th sample, and Gcls j For the category of the j-th labeled instance, Gbox j This represents the position of the bounding box for the j-th annotation instance.

[0038] Sample image im base_i and corresponding labeled data base_iTo create training sample data pairs, first, im base_i The input model, as shown in model structure diagram 1), yields the predicted outputs of the three branches of the model as Pboxes. k Pcls k and Pscore k k represents the index of the predicted instance result of the model for image i. The labeled data and output information are then processed through a Pbox. k and Gbox j The Intersection over Union (IOU) is used for matching. If the IOU is greater than 0.5, a matching pair is constructed to determine the positive sample. Let p be the IOU. k =1, and the remaining predicted outputs are used as negative samples (since there are many negative samples, in order to maintain the balance between positive and negative samples, the negative samples are randomly sampled to maintain the ratio of positive to negative samples of 1:1), denoted as p k =0, specifically for the matched positive samples, the pair of Pboxes is set to 0. k and Gbox j The cross-union ratio (CUIR) is used as the ground truth corresponding to the box score at that time, i.e., the Pscore. k The regression target value is denoted as GPiou. kj .

[0039] Its model loss function is

[0040] L=E(Pcls k ,Gcls j )+Reg(Pbox k Gbox j )*p k +Reg(Pscore k ,GPiou kj )*p k

[0041] Here, E represents the classification loss function, Reg represents the regression loss function, and p... k The regression loss is calculated only for positive samples. During training, stochastic gradient descent is used for optimization, with the learning rate set to 0.001. This results in a model, model_base, trained on the base dataset.

[0042] 3) Pseudo-tag generation section

[0043] Unlabeled data is represented as Imnovel. n Where n is the ordinal number of the unlabeled image, the model model_base is used to predict its value, and the prediction result is denoted as PNcls. m PNbox m PNscore m, where m represents the index of the model prediction result instance for the unlabeled image n, here taken as PNcls. m *PNscore m Detection results with a value greater than or equal to 0.8 are used as pseudo-labels. This ensures both the reliability of the category and the accuracy of the target box location of the pseudo-label. These reliable pseudo-labels are denoted as PGNcls. m PGNbox m PGNscore m For PNcls m *PNscore m Prediction results with a value of 0.5 or less are not recorded and are treated as background. The remaining prediction outputs are considered intermediate results, and pseudo-labels are output normally with special markings for subsequent loss function calculations. These reliable labels and intermediate result labels (non-background results) constitute the pseudo-label sample dataset.

[0044] 4) Retraining section:

[0045] First, the basic sample dataset and the pseudo-label sample dataset are merged and reconstructed. The basic sample dataset is randomly selected, and the number is twice the size of the aforementioned pseudo-label sample dataset. This sample is then mixed with the aforementioned pseudo-label sample dataset to form the retraining sample dataset.

[0046] Then, training is performed based on the retraining sample dataset. The overall process is similar to the training process on the basic dataset. The difference is that when calculating the loss function, the loss error corresponding to the intermediate result instances marked by pseudo-labels is set to zero in the loss function. That is, these instances do not contribute to the update of model parameters, so as to ensure the ability of the new model to express intermediate results.

[0047] The model was trained in this way, and the best model parameters on the retrained sample dataset (mAp50, a general test metric for object detection) were used as the new model parameters.

[0048] The pseudo-label algorithm of this invention adds multi-level threshold label processing to inherit the ability of the base model to output intermediate results on new datasets, thereby maintaining the output diversity of the new model;

[0049] This invention filters and extracts pseudo-labels by adding a regression position quality judgment branch and combining it with the category probability output. This maintains the accuracy of the pseudo-label category while ensuring the accuracy of the pseudo-label position.

[0050] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A pseudo-label based object detection training method, characterized in that, The classification output of the method is processed in multiple stages, and is generally divided into three levels, i.e., a class output probability greater than or equal to a first threshold value is taken as a pseudo label of the class of the target, a class output probability value greater than a second threshold value and less than the first threshold value is marked as a special class, and a class output probability value less than or equal to the second threshold value is taken as a background class; in the optimization of the loss function, the special class does not contribute to the loss function, that is, the output ability of the model for these uncertain targets is retained; The output of the target detection algorithm model includes a classification branch, a regression branch and a regression box reliability judgment branch, on the basis of the output of the regression branch, the IoU value of the regression branch output result and the labeled box is calculated, and the IoU value is taken as the learning target of the regression box reliability judgment branch, and the corresponding regression branch output is more accurate when the output value of the regression box reliability judgment branch is high. Wherein, The target detection algorithm model includes a backbone network and an RCNN Head, an input image is subjected to feature extraction by the backbone network, and then subjected to RoIAlign operation, and then enters the RCNN Head, and then subjected to convolution and two fully connected layers, and then outputs a classification branch of a target class, a regression branch of a target position and a regression box reliability judgment branch, and the regression box reliability judgment branch is a box score output branch with a judgment target position prediction accuracy. 2.The pseudo-label based object detection training method of claim 1, wherein, The first threshold value is 0.

8. 3.The pseudo-label based object detection training method of claim 1, wherein, The second threshold value is 0.

5.

4. The pseudo-label based object detection training method of any one of claims 1-3, wherein, The method comprises a basic data set training part: a labeled basic training data sample set is referred to as D base , im base_i is a corresponding sample image, label base_i is corresponding artificial labeling information, i is a sample index, Gcls j is a class of the jth labeled instance, Gbox j is a box position of the jth labeled instance N , j is an index of a labeled instance corresponding to the ith sample, Gcls N is a class of the jth labeled instance, Gbox j is a box position of the jth labeled instance j ​ Sample image im base_i and corresponding label data label base_i The training sample data pair is composed of im base_i First, input the model to obtain the prediction output of the three branches of the model as Pbox k , Pcls k and Pscore k , k represents the output prediction instance result of the model for the image i, and then the label information and the output information are matched through the intersection over union IoU of Pbox k and Gbox j , if the IOU is greater than a third threshold, a matching pair is constructed to determine the positive sample, denoted as p k =1, and the remaining prediction output is a negative sample, denoted as p k =0; for the matched positive sample, the value of the intersection over union of Pbox k and Gbox j is taken as the ground truth corresponding to the box score at this time, that is, the regression target value of Pscore k , denoted as GPiou kj ; the model loss function is L=E(Pcls k ,Gcls j )+Reg(Pbox k ,Gbox j )*p k +Reg(Pscore k ,GPiou kj )*p k Here, E represents a classification loss function, Reg represents a regression loss function, and p k only calculates the regression loss for the positive sample; thus, a model based on the basic data set, model_base, is obtained through training of the basic data set. 5.The pseudo-label based object detection training method of claim 4, wherein, The third threshold value is 0.

5. 6.The pseudo-label based object detection training method of claim 4, wherein, When training the basic data set, the random gradient descent optimization method is used, and the parameter learning rate is set to 0.

001.

7. The pseudo-label based object detection training method of claim 4, wherein, The method further comprises a pseudo-label generation part: representing the unlabeled data as Imnovel n , n is the sequence number of the unlabeled image, and the model model_base is used for prediction, and the prediction result is recorded as PNcls m , PNbox m , PNscore m , m represents the sequence number of the model prediction result instance of the unlabeled image n, PNcls m *PNscore m , and the detection result greater than or equal to the first threshold value is used as a pseudo label, and these pseudo labels are recorded as PGNcls m , PGNbox m , PGNscore m , the prediction result PNcls m *PNscore m less than or equal to the second threshold value is not recorded, that is, as a background class; the remaining prediction output is used as an intermediate result, the pseudo label is normally output, and the pseudo label is specially marked for subsequent calculation of a loss function; and the results of these reliable labels and intermediate result labels constitute a pseudo label sample data set. 8.The pseudo-label based object detection training method of claim 7, wherein, The method further includes a retraining part: first, the basic sample data set and the pseudo label sample data set are fused and reconstructed, the basic sample data set is randomly extracted, the number is twice the size of the aforementioned pseudo label sample data set, and the basic sample data set is mixed with the aforementioned pseudo label sample data set to form a retraining sample data set; then based on the retraining sample data set, in the calculation of the loss function, for the intermediate result instance part marked by the pseudo label, the corresponding loss error in the loss function is all set to zero, that is, these instances do not contribute to the model parameter update. 9.The pseudo-label based object detection training method of claim 8, wherein, The best model parameter of the target detection general test index mAp50 on the retraining sample data set is taken as the new model parameter.

Images