Semi-supervised object detection model training method and system based on hybrid learning framework

Abstract

The present invention provides a method and system for training a semi-supervised target detection model based on a hybrid learning framework. The method comprises: obtaining a teacher model and a student model; obtaining a training set; performing a strong enhancement operation on any labeled data and inputting it into the student model to obtain a labeled detection result; calculating a supervised loss; performing a weak enhancement operation and a strong enhancement operation on the unlabeled data and inputting it into the teacher model for detection to generate a pseudo label, and then inputting it into the student model for detection to obtain an unlabeled detection result; extracting a pseudo negative label; calculating the pseudo label loss and the pseudo negative label loss to form an unsupervised loss; and iteratively updating the parameters of the student model and the teacher model. In the hybrid learning strategy provided by the present invention, positive learning enhances the model's positioning accuracy for the target; while negative learning effectively improves the utilization rate of unlabeled data, it reduces the propagation of noise in the pseudo label, and ultimately significantly improves the training efficiency and the accuracy of model detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and machine learning technologies, and in particular to a semi-supervised target detection model training method and system based on a hybrid learning framework. Background Art

[0002] Object detection plays a key role in the field of computer vision and is widely used in many fields, including security monitoring, autonomous driving, and image recognition. With the rise and development of deep learning technology, object detection technology has made great progress.

[0003] In practical application scenarios, a major challenge facing target detection is the high manpower and time costs required for data labeling. Accurately labeling each target requires careful manual identification, and the labeling task is particularly arduous when faced with large-scale data sets. Based on this, semi-supervised learning has become an important means to solve the problem of scarce labeled data. Among them, pseudo-label-based methods are one of the mainstream strategies for semi-supervised target detection. A common practice is to train a model to generate pseudo-labels for unlabeled data, filter out high-quality pseudo-labels by setting a threshold, and then combine the losses of labeled and unlabeled data to train the model. There are also two-stage frameworks that use pseudo-labels to simultaneously train related network modules to solve the problem of pseudo-label bias; there are also dual-threshold strategies that classify pseudo-labels and use the originally discarded data to supervise the regression task.

[0004] However, current semi-supervised object detection methods still have many limitations. In classification tasks, since the true category of unlabeled data is unknown, the generated pseudo-labels inevitably contain noise. This noise can convey erroneous information during model training, leading to reduced model performance. Furthermore, methods that have proven effective in image classification, such as using pseudo-negative labels to reduce errors, have yet to be applied in semi-supervised object detection, creating a gap in current research. For bounding box regression tasks, some traditional loss calculation methods assume the independence of bounding box coordinates, which is inconsistent with reality. Others, while linking coordinate information through the intersection-over-union (IoU) calculation, fail to fully consider positional relationships. Even improved methods often fail to prioritize high-confidence predictions, which have been shown to be crucial for improving model performance in supervised classification. Consequently, existing methods struggle to meet the growing demand for accurate detection. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a semi-supervised target detection model training method and system based on a hybrid learning framework.

[0006] To achieve the aforementioned object of the invention, the technical solutions adopted by the present invention include:

[0007] In a first aspect, the present invention provides a semi-supervised target detection model training method based on a hybrid learning framework, which comprises:

[0008] Obtain a first basic model and a second basic model with the same structure and use them as a teacher model and a student model respectively; obtain a training set, wherein the training set includes labeled data and its corresponding true value labels, and unlabeled data;

[0009] Performing a strong enhancement operation on the labeled data to obtain a labeled enhancement result, inputting the labeled enhancement result into the student model for detection to obtain a labeled detection result;

[0010] Calculating a supervised loss based on the labeled detection results and the true value labels;

[0011] Performing a weak enhancement operation and a strong enhancement operation on the unlabeled data to obtain an unlabeled weak enhancement result and an unlabeled strong enhancement result respectively;

[0012] Input the unlabeled weak enhancement result into the teacher model for detection to generate a pseudo label; input the unlabeled strong enhancement result into the student model for detection to obtain an unlabeled detection result;

[0013] Dividing the pseudo labels into reliable pseudo labels and uncertain pseudo labels, extracting pseudo negative labels from the multiple uncertain pseudo labels, wherein the classification reliability of the pseudo negative labels is higher than the average classification reliability of the uncertain pseudo labels;

[0014] Calculating a pseudo label loss based on the consistency between the reliable pseudo label and the unlabeled detection result, and calculating a pseudo negative label loss based on the negative consistency between the pseudo negative label and the unlabeled detection result, wherein the pseudo label loss and the pseudo negative label loss constitute an unsupervised loss;

[0015] Based on the supervised loss and the unsupervised loss, the parameters of the student model are iteratively updated, and based on the parameters of the student model, the parameters of the teacher model are iteratively updated.

[0016] In a second aspect, the present invention further provides a semi-supervised target detection model training system based on a hybrid learning framework, which comprises:

[0017] A model data module is configured to obtain a first basic model and a second basic model having the same structure and use them as a teacher model and a student model, respectively; obtain a training set, wherein the training set includes labeled data and its corresponding true value labels, and unlabeled data;

[0018] A label enhancement module is used to perform a strong enhancement operation on the labeled data to obtain a labeled enhancement result, and to perform a weak enhancement operation and a strong enhancement operation on the unlabeled data to obtain an unlabeled weak enhancement result and an unlabeled strong enhancement result respectively;

[0019] A supervised loss module is used to input the labeled enhancement result into the student model for detection, obtain a labeled detection result, and calculate a supervised loss based on the labeled detection result and the true value label;

[0020] An unsupervised loss module is configured to input the unlabeled weak enhancement result into the teacher model for detection to generate a pseudo label; input the unlabeled strong enhancement result into the student model for detection to obtain an unlabeled detection result, and calculate the unsupervised loss based on the pseudo label and the unlabeled detection result;

[0021] an iterative update module, configured to iteratively update the parameters of the student model based on the supervised loss and the unsupervised loss, and iteratively update the parameters of the teacher model using a moving average method based on the parameters of the student model;

[0022] Among them, the unsupervised loss module divides the pseudo labels into reliable pseudo labels and uncertain pseudo labels, extracts pseudo negative labels from multiple uncertain pseudo labels, the classification reliability of the pseudo negative labels is higher than the average classification reliability of the uncertain pseudo labels, calculates the pseudo label loss based on the consistency of the reliable pseudo labels and the unlabeled detection results, and calculates the pseudo negative label loss based on the negative consistency of the pseudo negative labels and the unlabeled detection results. The pseudo label loss and the pseudo negative label loss constitute the unsupervised loss.

[0023] In a third aspect, the present invention also provides a semi-supervised target detection model obtained by training using the above training method.

[0024] In a fourth aspect, the present invention also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is run by the processor, it executes the steps in the above-mentioned model training method or runs the above-mentioned semi-supervised target detection model.

[0025] In a fifth aspect, the present invention also provides a readable storage medium, in which a computer program is stored. When the computer program is run, the steps in the above-mentioned model training method are executed or the above-mentioned semi-supervised target detection model is run.

[0026] Based on the above technical solution, compared with the prior art, the beneficial effects of the present invention include at least:

[0027] In the hybrid learning strategy provided by the present invention, positive learning focuses on improving the bounding box positioning capability, which can obtain more incentives from good bounding box regression predictions, thereby enhancing the model's positioning accuracy of the target; negative learning focuses on the object classification link, and by simultaneously using pseudo-negative labels and pseudo-labels to calculate pseudo-negative classification loss and classification loss, it effectively improves the utilization rate of unlabeled data while reducing the propagation of noise in pseudo-labels, ultimately significantly improving training efficiency and model detection accuracy.

[0028] The above description is only an overview of the technical solution of the present invention. In order to enable those skilled in the art to more clearly understand the technical means of this application and implement them according to the contents of the specification, the following is an explanation of the preferred embodiments of the present invention with detailed drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 1 is a flow chart of a semi-supervised target detection model training method provided by a typical embodiment of the present invention;

[0030] Figure 2 It is a system architecture diagram of a semi-supervised target detection model training method provided by a typical implementation case of the present invention. DETAILED DESCRIPTION

[0031] In view of the shortcomings of the prior art, the inventors of this case, after long-term research and extensive practice, have proposed the technical solution of the present invention. The following will further explain this technical solution, its implementation process and principles.

[0032] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0033] Moreover, relational terms such as “first” and “second” are merely used to distinguish one component or method step from another with the same name, but do not necessarily require or imply any actual relationship or order between these components or method steps.

[0034] See also Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a method for training a semi-supervised target detection model based on a hybrid learning framework, wherein the target detection model is used at least for image target detection and classification, and comprises the following steps:

[0035] 1) Obtaining a first basic model and a second basic model with the same structure as the teacher model and the student model respectively; obtaining a training set, wherein the training set includes labeled data and its corresponding true value labels, and unlabeled data;

[0036] 2) performing a strong enhancement operation on any of the labeled data to obtain a labeled enhancement result, inputting the labeled enhancement result into the student model for detection to obtain a labeled detection result;

[0037] 3) Calculating a supervised loss based on the labeled detection results and the true value labels corresponding to the labeled data;

[0038] 4) performing a weak enhancement operation and a strong enhancement operation on any of the unlabeled data to obtain an unlabeled weak enhancement result and an unlabeled strong enhancement result respectively;

[0039] 5) Inputting the unlabeled weak enhancement result into the teacher model for detection to generate a pseudo label; inputting the unlabeled strong enhancement result into the student model for detection to obtain an unlabeled detection result;

[0040] 6) dividing the pseudo labels into reliable pseudo labels and uncertain pseudo labels according to their scores, extracting pseudo negative labels from the multiple uncertain pseudo labels, wherein the classification reliability of the pseudo negative labels is higher than the average classification reliability of the uncertain pseudo labels;

[0041] 7) Calculating the pseudo label loss of the student model based on the consistency between the reliable pseudo label and the unlabeled detection result, and calculating the pseudo negative label loss of the student model based on the negative consistency between the pseudo negative label and the unlabeled detection result, wherein the pseudo label loss and the pseudo negative label loss constitute the unsupervised loss;

[0042] 8) Iteratively updating the parameters of the student model based on the supervised loss and the unsupervised loss, and iteratively updating the parameters of the teacher model based on the parameters of the student model using a moving average method.

[0043] The student and teacher models can preferably be pre-trained using the labeled data and true value labels, or directly obtain the target detection model obtained by preliminary training in other methods or programs. The true value label is also called the ground truth label. In addition, for the convenience of demonstration, the embodiment of the present invention provides Figure 2 Rotated left.

[0044] The present invention proposes a hybrid learning training method that combines positive learning and negative learning. Positive learning enhances the learning of bounding box regression from high-confidence samples by constructing a new regression loss function with an incentive term; negative learning improves the classification performance of target detection by converting classification-reliable uncertain pseudo-labels into corresponding pseudo-negative labels and using the pseudo-negative labels together with the pseudo-labels. The present invention also theoretically proves the superiority of this learning strategy. Through the above-mentioned hybrid learning framework, the potential of pseudo-labels is tapped, and the waste of information caused by discarding low-confidence pseudo-labels is avoided, thereby significantly improving the training efficiency of the model and the accuracy of target detection.

[0045] As some typical application examples, the above training method can be implemented using the following steps:

[0046] S1. Create and independently initialize two structurally consistent basic models as teacher and student models respectively.

[0047] S2. Perform strong enhancement operations on labeled data, and weak enhancement and strong enhancement operations on unlabeled data respectively.

[0048] S3. Input the strongly enhanced labeled data into the student model respectively, and obtain the supervised loss value of the student model by calculating the model prediction and the corresponding true value label.

[0049] S4. For unlabeled data, the weakly enhanced data is input into the teacher model to generate pseudo labels, and the strongly enhanced data is input into the student model to generate predictions.

[0050] S5. Generate corresponding representative pseudo-negative labels based on the pseudo-labels generated above, combine the pseudo-labels and pseudo-negative labels as the learning target for strong enhancement data prediction and calculate the unsupervised loss value.

[0051] S6. Use gradient descent to update the student model parameters, and use exponential sliding average to update the teacher model parameters based on the student model parameters. Repeat S2-S5 until convergence.

[0052] The process can be summarized as follows: create and initialize teacher and student models with the same structure; strongly enhance the labeled data and weakly and strongly enhance the unlabeled data respectively; calculate the supervised loss of the student model based on the strongly enhanced labeled data and the true value labels; use the teacher model to generate pseudo labels and corresponding pseudo negative labels for the weakly enhanced unlabeled data; use the student model to generate predictions for the strongly enhanced unlabeled data, and calculate the unsupervised loss by combining the pseudo labels and pseudo negative labels; and update the parameters of the two models.

[0053] Regarding some specific technical details, such as the calculation of supervised loss, in some implementation schemes, the calculation method of the supervised loss can be expressed as:

[0054] ;

[0055] in, represents the supervised loss, represents the supervised classification loss value, represents the supervised bounding box regression loss value, Represents the supervised confidence loss value.

[0056] More specifically, in some embodiments, the loss functions of the supervised classification loss value and the supervised confidence loss value are respectively:

[0057] ;

[0058] in, represents the cross entropy loss; Represents the classification result in the true value label; Represents the classification prediction part of the labeled detection result; Represents the confidence annotation result in the true value label; represents the confidence prediction part of the labeled detection result, represents the bounding box prediction part in the labeled detection result, Annotate the bounding box coordinates in the true value label, Represents the bounding box prediction part and the bounding box coordinates are marked The intersection-over-union ratio, Represents the bounding box prediction part and the bounding box coordinates are marked distance, Represents the bounding box prediction part and the bounding box coordinates are marked aspect ratio consistency.

[0059] More detailed, The calculation formula can be:

[0060] ;

[0061] Represents the predicted box and the real frame The distance is calculated as follows:

[0062] ;

[0063] in is the predicted box and the real frame The Euclidean distance between the center points of Is to cover the prediction box and the real frame The minimum diagonal distance of the bounding box. It is a parameter used to control the range of the interval. exist[ , ] interval, By changing the gradient from Change to To increase the gradient of back propagation, thereby rewarding predictions that are close to correct and enhancing the model's learning from samples with good regression;

[0064] Represents the predicted box and the real frame Aspect ratio consistency, The calculation formula is:

[0065] ;

[0066] in , and Respectively represent the prediction boxes The width and height of and Represent the real boxes respectively width and height. is a parameter used to balance the proportions, defined as Similarly, when exist[ , ] interval, Will Replace with its tangent , increasing the back-propagated gradient to reward predictions that are close to correct, enhancing the model’s learning from samples with good regression.

[0067] In the hybrid learning training method combining positive learning and negative learning strategies proposed in the present invention, the positive learning strategy part is collaboratively composed of a loss function for supervised bounding box regression loss value and a loss function for unsupervised bounding box regression loss value.

[0068] In some embodiments, the unsupervised loss is calculated as:

[0069] ;

[0070] in, represents the unsupervised loss, represents the unsupervised classification loss value, represents the unsupervised bounding box regression loss value, represents the unsupervised confidence loss value, Represents the false negative classification loss value.

[0071] In some embodiments, the method may specifically include:

[0072] During the pseudo-label generation process, multiple confidence scores related to the pseudo-label are generated simultaneously. The confidence scores include: objectivity score , used to evaluate the reliability of the pseudo-label positioning and the classification score , used to evaluate the reliability of the pseudo-label classification and the pseudo-label score , used to evaluate the overall quality of the pseudo-labels.

[0073] As a specific implementation example, in the process of generating the pseudo-label, the teacher model will simultaneously generate three confidence scores of the pseudo-label, which will be used specifically for filtering the pseudo-label. The three confidence scores include: objectivity score , used to evaluate the reliability of pseudo-label positioning; classification score , used to evaluate the reliability of pseudo-label classification; pseudo-label score , used to evaluate the overall quality of pseudo-labels.

[0074] In some embodiments, the process of screening the pseudo-labels and extracting the pseudo-negative labels may specifically include:

[0075] Based on the confidence scores, the generated pseudo labels to be selected are divided into overall reliability, positioning reliability and classification reliability; all confidence scores of the overall reliability type are higher than a preset value, the objectivity score of the positioning reliability type is higher than a preset value, and the classification score is lower than a preset value; the objectivity score of the classification reliability type is lower than a preset value, and the classification score is higher than a preset value;

[0076] The overall reliability participates in calculating the unsupervised classification loss value, the unsupervised bounding box regression loss value and the unsupervised confidence loss value; the positioning reliability participates in calculating the unsupervised bounding box regression loss value and the unsupervised confidence loss value; the classification reliability participates in calculating the unsupervised confidence loss value and is used to generate the pseudo negative label.

[0077] In the above technical solution, pseudo labels are divided into two categories: reliable and uncertain. The uncertain pseudo labels are further divided into classification reliable uncertain pseudo labels and positioning reliable uncertain pseudo labels. The reliable pseudo labels are used to calculate unsupervised classification, regression and confidence loss. The classification reliable uncertain pseudo labels are converted into corresponding pseudo negative labels for calculating unsupervised pseudo negative classification loss. The positioning reliable uncertain pseudo labels are used to calculate unsupervised regression and confidence loss.

[0078] In some embodiments, the process of generating the pseudo-negative label specifically includes:

[0079] (i) Maintain a size of The matrix of Represents the total number of sample categories in the dataset;

[0080] (ii) recording each round of predictions of the student model on the labeled dataset;

[0081] (iii) Based on the classification results in the true value labels and the classification prediction part in the labeled data detection results, determine that in this round of prediction, the student model will classify The probability value of misclassification into other categories is accumulated into the matrix No. Rows where the category is the category corresponding to the true value label;

[0082] (iv) with the addition of the Matrix of rows Can be used as a selection matrix for pseudo-negative labels;

[0083] (v) When it is necessary to select the pseudo label When there are pseudo negative labels, in the matrix No. Find in the row The largest value, and this The category corresponding to the maximum value is the pseudo label representative pseudo negative labels.

[0084] In some embodiments, the loss function of the pseudo negative classification loss value is:

[0085] ;

[0086] in, represents the pseudo negative classification loss value, Represents the classification prediction results of some of the pseudo labels selected as pseudo negative labels; Represents the classification prediction part in the unlabeled detection result; Indicates that the pseudo negative label is filtered to satisfy all the pseudo labels and part, 、 、 are the corresponding preset values, Indicates that the pseudo-label score in the pseudo-label satisfies And classification score The part is used to calculate the pseudo negative classification loss value; Represents the negative cross entropy loss, i.e. The specific calculation formula can be:

[0087] ;

[0088] in is a binary vector indicating which class is selected as the pseudo-label classification prediction The corresponding pseudo negative label, Indicates the category, specifically, when selecting categories as the current pseudo-label classification prediction When the pseudo negative label is The value of is 1, otherwise it is 0. Of course, in practical applications, the expression is not limited to 0 and 1.

[0089] In some embodiments, the loss function of the unsupervised confidence loss value is:

[0090] ;

[0091] in, represents the cross entropy loss; Represents the confidence prediction result in the pseudo label; Represents the confidence prediction part in the unlabeled detection result.

[0092] In some embodiments, the loss function of the unsupervised bounding box regression loss value is:

[0093] ;

[0094] in, The bounding box prediction part in the unlabeled detection result; The bounding box prediction result in the pseudo label. The calculation formula is the same as that in the supervised loss Keep consistent with Together they constitute the positive learning strategy part of the hybrid learning training method proposed in the present invention, while the negative learning strategy part is collaboratively constructed by the loss function for unsupervised classification and the pseudo-negative classification loss function.

[0095] In some embodiments, the loss function of the unsupervised classification loss value is:

[0096] ;

[0097] in, represents the cross entropy loss; Represents the classification prediction result in the pseudo label; Represents the classification prediction part in the unlabeled detection result; Represents the indicator function, when it satisfies When the conditions Output 1 if yes, otherwise output 0.

[0098] In some embodiments, the supervised loss and the unsupervised loss are linearly added as the total loss to iteratively update the student model and the teacher model. The calculation method of the total loss is expressed as:

[0099] ;

[0100] in, represents the total loss, It represents the balance coefficient, and its value range is between 1 and 0. It can be determined through conditional experiments. The present invention does not impose specific numerical restrictions on this. Finally, based on the parameters of the student model, the exponential moving average method is used to iteratively update the parameters of the teacher model. The specific student-teacher common parameter update method can refer to many existing technologies. It is not the key point of the present invention and is not specifically limited here.

[0101] In some embodiments, the iterations in the training method can be performed in batches, i.e., the training set is divided into multiple batches, each batch corresponding to one iteration. The number of data in the batch can be greater than one, or only one, in which case the training method is performed without batching.

[0102] The second part of the embodiment of the present invention further provides a semi-supervised target detection model training system based on a hybrid learning framework, which includes:

[0103] A model data module is configured to obtain a first basic model and a second basic model having the same structure and use them as a teacher model and a student model, respectively; obtain a training set, wherein the training set includes labeled data and its corresponding true value labels, and unlabeled data;

[0104] A label enhancement module is used to perform a strong enhancement operation on any of the labeled data to obtain a labeled enhancement result, and to perform a weak enhancement operation and a strong enhancement operation on any of the unlabeled data to obtain an unlabeled weak enhancement result and an unlabeled strong enhancement result respectively;

[0105] A supervised loss module is used to input the labeled enhancement result into the student model for detection, obtain a labeled detection result, and calculate a supervised loss based on the labeled detection result and the true value label corresponding to the labeled data;

[0106] An unsupervised loss module is configured to input the unlabeled weak enhancement result into the teacher model for detection to generate a pseudo label; input the unlabeled strong enhancement result into the student model for detection to obtain an unlabeled detection result, and calculate the unsupervised loss based on the pseudo label and the unlabeled detection result;

[0107] an iterative update module, configured to iteratively update the parameters of the student model based on the supervised loss and the unsupervised loss, and iteratively update the parameters of the teacher model using a moving average method based on the parameters of the student model;

[0108] Among them, the loss calculation strategy composed of the supervised loss module and the unsupervised loss module includes a positive learning part and a negative learning part; the positive learning part is used to calculate the supervised loss and the part of the pseudo-label loss related to target positioning in the unsupervised loss; the negative learning part is used to extract pseudo-negative labels from multiple pseudo-labels, and calculate the pseudo-negative label loss related to the target classification of the student model based on the negative consistency of the pseudo-negative labels and the unlabeled detection results, as well as calculate the part of the pseudo-label loss related to the target classification in the unsupervised loss; in the unsupervised loss module, the pseudo-label loss and the pseudo-negative label loss together constitute the unsupervised loss.

[0109] The technical solution of the present invention is further described in detail below through several embodiments and in conjunction with the accompanying drawings. However, the selected embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0110] Example 1

[0111] This embodiment illustrates a semi-supervised training method for a target detection model based on the above technical solution. The specific steps are as follows:

[0112] (1) Obtain a first base model and a second base model with the same structure and use them as the teacher model and student model respectively; obtain a training set consisting of a small amount of labeled data and a large amount of unlabeled data. Divide the labeled data and unlabeled data into batches of a specific size and input them into the two models in sequence for training.

[0113] (2) For a small batch of labeled data, a strong enhancement operation is performed on each image therein to obtain a labeled strong enhancement result, and the labeled strong enhancement results obtained in the batch are input into the student model for detection to obtain a detection result of the labeled data.

[0114] (3) After the above operations, a batch of labeled data is used as the input of the student model to generate the corresponding predictions. The supervised classification loss values ​​are then calculated for the model predictions and the corresponding ground truth labels. , supervised bounding box regression loss value and supervised confidence loss .

[0115] The loss functions of the supervised classification loss value and the supervised confidence loss value are respectively:

[0116] ;

[0117] in, represents the general cross entropy loss; Represents the classification result in the true value label; Represents the classification prediction part of the labeled data detection result; Represents the confidence annotation result in the true value label; Represents the confidence prediction part in the labeled data detection result.

[0118] The loss function of the supervised bounding box regression loss value is:

[0119] ;

[0120] in, The bounding box prediction part in the labeled data detection result; Annotate the bounding box coordinates in the ground truth label; Represents the predicted box and the real frame The calculation formula of the intersection-and-union ratio is:

[0121] ;

[0122] Represents the predicted box and the real frame The distance is calculated as follows:

[0123] ;

[0124] in is the predicted box and the real frame The Euclidean distance between the center points of Is to cover the prediction box and the real frame The minimum diagonal distance of the bounding box. It is a parameter used to control the range of the interval. exist[ , ] interval, By changing the gradient from Change to To increase the gradient of back propagation, thereby rewarding predictions that are close to correct and enhancing the model's learning from samples with good regression;

[0125] Represents the predicted box and the real frame Aspect ratio consistency, The calculation formula is:

[0126] ;

[0127] in , and Respectively represent the prediction boxes The width and height of and Represent the real boxes respectively width and height. is a parameter used to balance the proportions, defined as Similarly, when exist[ , ] interval, Will Replace with its tangent , increasing the back-propagation gradient to reward predictions close to correct, enhancing the model's learning from samples with good regression. In the hybrid learning training method proposed in this invention that combines positive and negative learning strategies, the positive learning strategy is composed of a loss function for supervised bounding box regression loss and a loss function for unsupervised bounding box regression loss.

[0128] Finally, the supervised loss calculation formula of the student model can be shown as follows:

[0129] ;

[0130] in, Represents the supervised loss value, which is the supervised classification loss value , supervised bounding box regression loss value and supervised confidence loss composition.

[0131] (4) For a small batch of unlabeled data, perform weak enhancement and strong enhancement operations on each image to obtain unlabeled weak enhancement results and strong label weak enhancement results. The unlabeled weak enhancement results obtained in the batch are input into the teacher model to generate pseudo labels for the unlabeled data. In the process of generating the pseudo labels, the teacher model simultaneously generates three confidence scores for the pseudo labels, including: objectivity score , used to evaluate the reliability of pseudo-label positioning; classification score , used to evaluate the reliability of pseudo-label classification; pseudo-label score , used to evaluate the overall quality of pseudo labels. These scores will be used specifically for filtering the pseudo labels.

[0132] (5) After the above operations, the obtained unlabeled strong enhancement result data batch is used as the input of the student model to generate the corresponding prediction, and then the unsupervised classification loss value is calculated for the model prediction and the corresponding pseudo label respectively. , unsupervised bounding box regression loss value , unsupervised confidence loss value and the false negative classification loss .

[0133] The loss function of the unsupervised confidence loss value is:

[0134] ;

[0135] in, represents the general cross entropy loss; Represents the confidence prediction result in the pseudo label; Represents the confidence prediction part in the unlabeled data detection result;

[0136] The loss function of the unsupervised bounding box regression loss value is:

[0137] ;

[0138] in, The bounding box prediction part in the unlabeled data detection result; The bounding box prediction result in the pseudo label; The calculation formula is the same as that in step (3) Keep consistent with Together they constitute the positive learning strategy part of the hybrid learning training method proposed in the present invention;

[0139] The loss function of the unsupervised classification loss value is:

[0140] ;

[0141] in, represents the general cross entropy loss; Represents the classification prediction result in the pseudo label; Represents the classification prediction part in the unlabeled data detection result; Represents the indicator function, when it satisfies When the conditions Output 1, otherwise output 0; Indicates the pseudo label score filtered out from the pseudo label The part is used to calculate the unsupervised classification loss value;

[0142] The loss function of the pseudo negative classification loss value is:

[0143] ;

[0144] in, Represents the classification prediction result in the pseudo label; Represents the classification prediction part in the unlabeled data detection result; Indicates that the pseudo-label score in the pseudo-label satisfies And classification score The part is used to calculate the pseudo negative classification loss value; Represents the negative cross entropy loss, which is calculated as:

[0145] ;

[0146] in is a binary vector indicating which class is selected as the pseudo-label classification prediction The corresponding pseudo negative label, when the first categories as the current pseudo-label classification prediction When the pseudo negative label is The value of is 1, otherwise it is 0; in the hybrid learning training method combining positive learning and negative learning strategies proposed in the present invention, the negative learning strategy is collaboratively constructed by the loss function for unsupervised classification and the pseudo-negative classification loss function.

[0147] The above steps calculate the pseudo negative classification loss, which requires the pseudo label classification prediction The process of selecting the corresponding pseudo negative label includes:

[0148] (i) Maintain a size of Matrix ,in Represents the total number of sample categories in the dataset.

[0149] (ii) Recording each round of predictions made by the student model on the labeled dataset.

[0150] (iii) Based on the classification results in the true value label (assuming the true category is Class) and the classification prediction part in the labeled data detection result, it can be determined that in this round of prediction, the student model will classify The probability value of misclassification into other categories is accumulated into the matrix No. OK.

[0151] (iv) The matrix No. The row shows at a global level that the student model classifies The probability of misclassification into other categories, so the matrix Can be used as a selection matrix for pseudo negative labels.

[0152] (v) When it is necessary to select the pseudo label When there are pseudo negative labels (assuming that the classification prediction result in the pseudo label is category ), just need to No. Find in the row The largest value. The category corresponding to the maximum value is the pseudo label representative pseudo negative labels.

[0153] Finally, the unsupervised loss calculation formula of the student model can be shown as follows:

[0154] ;

[0155] in, Represents the unsupervised loss value, which is composed of the unsupervised classification loss value , unsupervised bounding box regression loss value , unsupervised confidence loss value and the false negative classification loss composition.

[0156] (6) Based on the linear sum of the supervised loss value in step (3) and the unsupervised loss value in step (5), the parameters of the student model are updated using gradient descent. The calculation formula of the total loss value can be:

[0157] ;

[0158] represents the balance coefficient between supervised loss and unsupervised loss; based on the parameters of the student model, the parameters of the teacher model are iteratively updated using the moving average method. Steps (2) to (5) are repeated until the student model converges, thereby obtaining a model that can be used for target detection.

[0159] Example 2

[0160] This embodiment illustrates the comparison between the training method provided in the above embodiment 1 and the existing semi-supervised target detection model training method:

[0161] In this embodiment, YOLOv5s is selected as the detector, and ImageNet pre-training is not performed. In terms of data sets, in the PASCAL-VOC benchmark, the trainval data of VOC07 is set as a labeled data set, the trainval data of VOC12 is used as an unlabeled data set, and the VOC07 test is used as a test set to test the model performance; in the COCO-standard data set, 1%, 2%, 5%, and 10% of the images are randomly selected from the train2017 set as labeled data sets, and the remaining images are used as unlabeled data sets, and val2017 is used as a test set for verification. This embodiment uses the NVIDIA A40 GPU, and each iteration takes 32 images from the labeled and unlabeled data at a 1:1 ratio. In the training configuration, the balance coefficient is set to =1.0, the learning rate is fixed at 0.01, 、 Set them to 0.1 and 0.6 respectively. is 0.99, the number of pseudo negative labels Set to 3, the bounding box regression loss The value of 0.05 is used for data augmentation. Weak augmentation uses Mosaic, while strong augmentation includes multiple operations such as Mosaic and left-right flipping. The maximum number of training rounds is set to 300.

[0162] In the field of object detection research, the PASCAL VOC and MS COCO datasets are the most commonly used benchmark datasets. Due to their rich annotation information and highly adaptable task settings, they have become important testing platforms for semi-supervised object detection research. This example uses the PASCAL VOC and MS COCO datasets for experiments.

[0163] Tables 1 and 2 below show the comparison results of the semi-supervised object detection method described in this example and existing semi-supervised object detection methods on the MS COCO and PASCAL VOC datasets.

[0164] Table 1 Comparison results with existing training methods under MS COCO dataset and YOLOv5s network structure

[0165]

[0166] Table 2 Comparison results with existing training methods under PASCAL VOC dataset and YOLOv5s network structure

[0167]

[0168] In Table 1 and Table 2 above, AP is used in this embodiment. 50 and AP 50:95 As the evaluation index of the model. Among them, Average Precision (AP) is used to measure the detection ability of the model on a single category, while Mean Average Precision (mAP) is a comprehensive evaluation of the detection performance of all categories. To calculate mAP, you must first set the intersection-over-union (IoU) threshold to determine the matching relationship between the detection results and the true annotations. Taking the PASCAL VOC dataset as an example, the IoU threshold is usually set to 0.5, and the mAP obtained at this time is recorded as AP. 50 During the calculation process, all detection results are first sorted from high to low according to the confidence level; if the IoU between the detection box and the true target exceeds the set threshold, it is considered a correct detection; then, the precision-recall curve is calculated based on all prediction results, and the area under the curve is calculated as the AP value of the category. Finally, the AP value of all categories is averaged to get AP. 50 In addition, AP 50:95 The mAP is calculated multiple times from IoU=0.5 to IoU=0.95 (with a step size of 0.05) and averaged to obtain the result.

[0169] Table 1 presents the results of experiments conducted on the COCO-standard dataset at four different annotation scales, using 1%, 2%, 5%, and 10% of the images randomly sampled as labeled datasets and the remaining images as unlabeled datasets. The supervised training model (generally referred to as "Supervised" in the field) represents a baseline method that performs supervised training only on the corresponding proportion of labeled data. The remaining comparison methods are all names of models published in existing international journals or conferences. The English name of the unbiased education model is Unbiased Teacher, the English name of the unique education model is OneTeacher, and the English name of the efficient education model is Efficient Teacher. The remaining models are known abbreviations of existing models. The "hybrid learning framework" is the training method provided in this embodiment.

[0170] Table 2 shows the results on the PASCA-VOC dataset. Similarly, the supervised training model represents a method based entirely on supervised learning. The remaining comparison methods are from existing international journals or conferences. The "hybrid learning framework" is the training method provided in this embodiment. The underlying network used in the "hybrid learning framework" in Tables 1 and 2 is YOLOv5s. As can be seen from Tables 1 and 2, the recognition method described in this embodiment effectively improves the training efficiency and accuracy of the semi-supervised object detection model.

[0171] Example 3

[0172] This embodiment compares the training method provided in the above embodiment 1 with more existing semi-supervised target detection methods:

[0173] Table 3 below shows the comparison results of the semi-supervised object detection training algorithm provided in this example and the existing semi-supervised object detection training method on the 10% MS COCO dataset (10% of the images are extracted as the labeled dataset and the remaining images are used as the unlabeled dataset) and the PASCA VOC dataset, using YOLOv5l as the base model. 50:95As a model evaluation metric, the supervised training model represents a supervised learning method using only labeled data. The remaining comparison methods are from existing international journals or conferences. Model names that overlap with those in Tables 1 and 2 above have the same translation or abbreviation. Other examples include the instantaneous teaching model, which is called Instant Teaching; the humble teaching model, which is called Humble Teacher; the counter-interference teaching model, which is called CombatingNoise; the flexible teaching model, which is called Soft Teacher; the reflective teaching model, which is called RethinkingPse; the active teaching model, which is called Active Teacher; and the label competition model, which is called LabelMatch. The "hybrid learning framework" refers to the method described in this embodiment. As shown in Tables 1, 2, and 3, the method described in this embodiment effectively improves the performance of the semi-supervised object detection model under two different network structures.

[0174] Table 3 Comparison results with existing training methods under MS COCO and PASCA VOC datasets and YOLOv5l network structure

[0175]

[0176] Based on the above embodiments, it is clear that in the hybrid learning training method provided by the embodiments of the present invention, positive learning focuses on improving bounding box positioning capabilities, thereby enhancing the model's accuracy in locating the target by obtaining more incentives from well-regressed predictions. Negative learning focuses on object classification. By simultaneously using pseudo-negative labels and pseudo-labels, negative learning effectively improves the utilization of unlabeled data while reducing the propagation of noise in pseudo-labels, ultimately achieving improved training efficiency.

[0177] This embodiment also provides an electronic device, comprising: one or more processors; and a memory, wherein the memory is used to store one or more executable instructions; the one or more processors are configured to execute the one or more executable instructions to perform the training steps described in this embodiment or run the semi-supervised target detection model provided by an embodiment of the present invention.

[0178] The specific implementation means of the present invention can be a system, method and / or computer program product. The computer program product can include a computer-readable storage medium, which is loaded with computer-readable program instructions for causing a processor to implement various aspects of the present invention. The computer-readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. The computer-readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the above.

[0179] It should be noted that although the above describes the various steps in a specific order, it does not mean that the steps must be performed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0180] It should be understood that the embodiments described above are some, rather than all, embodiments of the present invention. The detailed description of the embodiments of the present invention is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are also within the scope of protection of the present invention.

Claims

1. A method for training a semi-supervised object detection model based on a hybrid learning framework, wherein the object detection model is used for at least image object detection and classification, characterized in that: include: Obtain a first basic model and a second basic model with the same structure and use them as a teacher model and a student model respectively; obtain a training set, wherein the training set includes labeled data and its corresponding true value labels, and unlabeled data, wherein the labeled data and the unlabeled data are both images; Performing a strong enhancement operation on any of the labeled data to obtain a labeled enhancement result, inputting the labeled enhancement result into the student model for detection to obtain a labeled detection result; Calculating a supervised loss based on the labeled detection results and the true value labels corresponding to the labeled data; Perform a weak enhancement operation and a strong enhancement operation on any of the unlabeled data to obtain an unlabeled weak enhancement result and an unlabeled strong enhancement result respectively; Inputting the unlabeled weak enhancement result into the teacher model for detection to generate a pseudo label; Inputting the unlabeled strong enhancement result into the student model for detection to obtain an unlabeled detection result; Dividing the pseudo labels into reliable pseudo labels and uncertain pseudo labels according to their scores, extracting pseudo negative labels from the multiple uncertain pseudo labels, wherein the classification reliability of the pseudo negative labels is higher than the average classification reliability of the uncertain pseudo labels; The pseudo label loss of the student model is calculated based on the consistency of the reliable pseudo label and the unlabeled detection result, and the pseudo negative label loss of the student model is calculated based on the negative consistency of the pseudo negative label and the unlabeled detection result. The pseudo label loss and the pseudo negative label loss constitute the unsupervised loss, and the calculation method of the unsupervised loss is expressed as: ； in, represents the unsupervised loss, represents the unsupervised classification loss value, represents the unsupervised bounding box regression loss value, represents the unsupervised confidence loss value, Represents the false negative classification loss value; The loss function of the pseudo negative classification loss value is: ； in, represents the pseudo negative classification loss value, Represents the classification prediction results of some of the pseudo labels selected as pseudo negative labels; Represents the classification prediction part in the unlabeled detection result; Indicates that the pseudo negative label is filtered as all the pseudo labels and part, 、 、 are the corresponding preset values ​​respectively; represents the negative cross entropy loss; The linear sum of the supervised loss and the unsupervised loss is used as the total loss, the parameters of the student model are iteratively updated, and based on the parameters of the student model, the parameters of the teacher model are iteratively updated using a moving average method.

2. The semi-supervised target detection model training method according to claim 1, characterized in that The calculation method of the supervised loss is expressed as: ； in, represents the supervised loss, represents the supervised classification loss value, represents the supervised bounding box regression loss value, Represents the supervised confidence loss value; And / or, the total loss is calculated as follows: ； in, represents the total loss, Indicates the balance coefficient.

3. The semi-supervised target detection model training method according to claim 2, characterized in that The loss functions of the supervised classification loss value and the supervised confidence loss value are respectively: ； in, represents the cross entropy loss; Represents the classification result in the true value label; Represents the classification prediction part of the labeled detection result; Represents the confidence annotation result in the true value label; represents the confidence prediction part of the labeled detection result, represents the bounding box prediction part in the labeled detection result, Annotate the bounding box coordinates in the true value label, Represents the bounding box prediction part and the bounding box coordinates are marked The intersection-over-union ratio, Represents the bounding box prediction part and the bounding box coordinates are marked distance, Represents the bounding box prediction part and the bounding box coordinates are marked aspect ratio consistency.

4. The semi-supervised target detection model training method according to claim 1, characterized in that Specifically include: During the pseudo-label generation process, multiple confidence scores related to the pseudo-label are generated simultaneously. The confidence scores include: objectivity score , used to evaluate the reliability of the pseudo-label positioning and the classification score , used to evaluate the reliability of the pseudo-label classification and the pseudo-label score , used to evaluate the overall quality of the pseudo-labels.

5. The semi-supervised target detection model training method according to claim 4, characterized in that: The process of screening the pseudo labels and extracting the pseudo negative labels specifically includes: Based on the confidence scores, the generated pseudo labels to be selected are divided into overall reliability, positioning reliability and classification reliability, and the positioning reliability and classification reliability belong to the uncertain pseudo labels; all confidence scores of the overall reliability type are higher than a preset value, the objectivity score of the positioning reliability type is higher than a preset value, and the classification score is lower than a preset value, and the objectivity score of the classification reliability type is lower than a preset value, and the classification score is higher than a preset value; The overall reliability participates in calculating the unsupervised classification loss value, the unsupervised bounding box regression loss value and the unsupervised confidence loss value; the positioning reliability participates in calculating the unsupervised bounding box regression loss value and the unsupervised confidence loss value; the classification reliability participates in calculating the unsupervised confidence loss value and is used to generate the pseudo negative label.

6. The semi-supervised target detection model training method according to claim 5, characterized in that The generation process of the pseudo negative label specifically includes: (i) Maintain a size of Matrix ,in Represents the total number of sample categories in the dataset; (ii) recording each round of predictions of the student model on the labeled dataset; (iii) Based on the classification results in the true value labels and the classification prediction part in the labeled data detection results, determine that in this round of prediction, the student model will classify The probability value of misclassification into other categories is accumulated into the matrix No. Rows where the category is the category corresponding to the true value label; (iv) with the addition of the Matrix of rows Can be used as a selection matrix for pseudo-negative labels; (v) When it is necessary to select the pseudo label When there are pseudo negative labels, in the matrix No. Find in the row The largest value, and this The category corresponding to the maximum value is the pseudo label representative pseudo negative labels.

7. The semi-supervised target detection model training method according to claim 1, characterized in that The loss function of the unsupervised confidence loss value is: ； in, represents the cross entropy loss; Represents the confidence prediction result in the pseudo label; Represents the confidence prediction part in the unlabeled detection result; And / or, the loss function of the unsupervised bounding box regression loss value is: ； in, The bounding box prediction part in the unlabeled detection result; The bounding box prediction result in the pseudo label; And / or, the loss function of the unsupervised classification loss value is: ； in, represents the cross entropy loss; Represents the classification prediction result in the pseudo label; Represents the classification prediction part in the unlabeled detection result; Represents the indicator function, when it satisfies When the conditions Output 1 if yes, otherwise output 0.

8. A semi-supervised target detection model training system based on a hybrid learning framework, used to implement the semi-supervised target detection model training method according to any one of claims 1 to 7, characterized in that: include: A model data module is configured to obtain a first basic model and a second basic model having the same structure and use them as a teacher model and a student model, respectively; obtain a training set, wherein the training set includes labeled data and its corresponding true value labels, and unlabeled data; A label enhancement module is used to perform a strong enhancement operation on any of the labeled data to obtain a labeled enhancement result, and to perform a weak enhancement operation and a strong enhancement operation on any of the unlabeled data to obtain an unlabeled weak enhancement result and an unlabeled strong enhancement result respectively; A supervised loss module is used to input the labeled enhancement result into the student model for detection, obtain a labeled detection result, and calculate a supervised loss based on the labeled detection result and the true value label corresponding to the labeled data; An unsupervised loss module, configured to input the unlabeled weak enhancement results into the teacher model for detection and generate pseudo labels; Inputting the unlabeled strong enhancement result into the student model for detection to obtain an unlabeled detection result, and calculating the unsupervised loss based on the pseudo label and the unlabeled detection result; an iterative update module, configured to iteratively update the parameters of the student model based on the supervised loss and the unsupervised loss, and iteratively update the parameters of the teacher model using a moving average method based on the parameters of the student model; The loss calculation strategy composed of the supervised loss module and the unsupervised loss module includes a positive learning part and a negative learning part; the positive learning part is used to calculate the supervised loss and the pseudo-label loss related to target positioning in the unsupervised loss; The negative learning part is used to extract pseudo negative labels from the multiple pseudo labels, and calculate the pseudo negative label loss of the student model related to the target classification based on the negative consistency of the pseudo negative label and the unlabeled detection result, as well as calculate the part of the pseudo label loss related to the target classification in the unsupervised loss; in the unsupervised loss module, the pseudo label loss and the pseudo negative label loss together constitute the unsupervised loss.

Images