Model training method and device, nonvolatile storage medium and electronic device

Abstract

This invention discloses a model training method, apparatus, non-volatile storage medium, and electronic device. The method includes: acquiring a first image set, wherein the first image set includes: multiple first images and pseudo-labels corresponding to each first image; analyzing the multiple first images in the first image set using a preset student model to determine a predicted label corresponding to each first image, wherein the preset student model is trained using machine learning on a second image set; calibrating the predicted labels based on the pseudo-labels to determine an optimized label corresponding to each first image; and training the preset student model using a third image set to obtain a target student model, wherein the third image set includes: multiple first images and optimized labels corresponding to each first image. This invention solves the technical problem of low accuracy in student models trained based on traditional pseudo-label methods.

Description

Technical Field

[0001] This invention relates to the field of machine learning, and more specifically, to a model training method, apparatus, non-volatile storage medium, and electronic device. Background Technology

[0002] Traditional manual annotation is a tedious, monotonous, and time-consuming process, forcing AI algorithm development into a labor-intensive task due to the large amount of data annotation work. Especially in complex business scenarios, prolonged manual annotation can easily lead to numerous mislabeling, omissions, and errors, often requiring developers to verify the data multiple times before it can be used to train the model, significantly impacting the model development schedule. Therefore, how to reduce the number of labeled samples required for training algorithms and how to enable automatic machine annotation have become critical bottlenecks that urgently need to be overcome.

[0003] The demands of the times have driven both industry and academia to conduct systematic research on semi-supervised learning. Compared to supervised learning, semi-supervised learning requires fewer labeled samples. It iteratively optimizes the detection algorithm and labeling method through model training-self-labeling-model training, ultimately achieving the goal of improving model performance by processing a large number of unlabeled samples.

[0004] In the field of semi-supervised learning, pseudo-labeling is a very basic method. It first trains a teacher model with a small amount of labeled data; then it uses the teacher model to label (pseudo-label) unlabeled data and assists the student model in continuously learning from all the pseudo-labeled unlabeled data; finally, the student model will outperform the teacher model and provide better prediction results.

[0005] However, a common criticism of traditional pseudo-labeling methods is that regardless of whether the pseudo-labels assigned to samples are correct or not, these labels have a significant impact on the student model. If a large number of unlabeled samples are assigned incorrect pseudo-labels, the student model will learn a large number of incorrect samples, severely affecting its learning outcomes. This is especially true in scenarios with small or long-tailed samples; if the teacher model is inaccurate, the resulting student model will also be very poor.

[0006] There is currently no effective solution to the problem of low accuracy in student models trained using the traditional pseudo-labeling method. Summary of the Invention

[0007] This invention provides a model training method, apparatus, non-volatile storage medium, and electronic device to at least solve the technical problem of low accuracy of student models trained based on traditional pseudo-labeling methods.

[0008] According to another aspect of the present invention, a model training method is also provided, comprising: acquiring a first image set, wherein the first image set includes: a plurality of first images and pseudo-labels corresponding to each first image; analyzing the plurality of first images in the first image set using a preset student model to determine a predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, the second image set including: a plurality of second images and true labels corresponding to each second image; calibrating the pseudo-labels based on the predicted labels to determine an optimized label corresponding to each first image; and training the preset student model using a third image set to obtain a target student model, wherein the third image set includes: a plurality of first images and the optimized label corresponding to each first image.

[0009] Optionally, before obtaining the first image set, the method further includes: obtaining a sample image set, wherein the sample image set includes: multiple first images without labels; adding pseudo-labels to the sample image set using a preset teacher model to determine the first image set, wherein the preset teacher model is trained by machine learning using the second image set.

[0010] Optionally, a preset teacher model is used to add pseudo-labels to the sample image set. Determining the first image set includes: using the preset teacher model to analyze multiple first images in the sample images, determining the pseudo-label and label confidence level corresponding to each first image; and determining the first image set based on the pseudo-labels whose label confidence level is greater than a preset confidence threshold.

[0011] Optionally, before determining the first image set based on the pseudo-labels whose label confidence is greater than a preset confidence threshold, the method further includes: identifying the label category of the pseudo-labels, wherein the label category includes at least: a first category and a second category; determining the preset confidence threshold corresponding to the label category, wherein the preset confidence threshold includes at least: a first confidence threshold corresponding to the first category and a second confidence threshold corresponding to the second category, wherein the first confidence threshold and the second confidence threshold are different.

[0012] Optionally, calibrating the pseudo-labels based on the predicted labels to determine the optimized label for each of the first images includes: determining the pseudo-label overlap rate between the pseudo-labels and the predicted labels, wherein the pseudo-labels are associated with bounding boxes in the first image, the predicted labels are associated with candidate boxes in the first image, and the pseudo-label overlap rate is the intersection-union ratio of the bounding boxes and the candidate boxes; determining the foreground box of the first image based on the pseudo-label overlap rate; and determining the predicted label associated with the foreground box based on the pseudo-labels.

[0013] Optionally, determining the foreground box of the first image based on the pseudo-label overlap rate includes: if the pseudo-label overlap rate is greater than a preset overlap threshold, determining the middle box between the label box and the candidate box as the foreground box, wherein the middle box is determined based on the median coordinate value of the label box and the candidate box; or if the pseudo-label overlap rate is not greater than the preset overlap threshold, determining the label box as the foreground box.

[0014] Optionally, determining the predicted label associated with the foreground box based on the pseudo label includes: determining the label confidence of the pseudo label relative to the annotation box as the label confidence of the predicted label relative to the foreground box.

[0015] According to another aspect of the present invention, a model training apparatus is also provided, comprising: an acquisition module for acquiring a first image set, wherein the first image set includes: a plurality of first images and pseudo-labels corresponding to each first image; an analysis module for analyzing the plurality of first images in the first image set using a preset student model to determine a predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, the second image set including: a plurality of second images and true labels corresponding to each second image; a determination module for calibrating the pseudo-labels based on the predicted labels to determine an optimized label corresponding to each first image; and a training module for training the preset student model using a third image set to obtain a target student model, wherein the third image set includes: a plurality of first images and the optimized label corresponding to each first image.

[0016] According to another aspect of the present invention, a non-volatile storage medium is also provided, the non-volatile storage medium being used to store a program, wherein the program controls the device where the non-volatile storage medium is located to execute the above-described model training method during runtime.

[0017] According to another aspect of the present invention, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the processor, wherein the program executes the above-described model training method when it runs.

[0018] In this embodiment of the invention, a first image set is obtained, comprising multiple first images and pseudo-labels corresponding to each first image; a preset student model is used to analyze the multiple first images in the first image set to determine the predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, which comprises multiple second images and true labels corresponding to each second image; the predicted labels are calibrated based on the pseudo-labels to determine the optimized label corresponding to each first image; the preset student model is trained using a third image set to obtain a target student model, wherein the third image set comprises multiple first images and optimized labels corresponding to each first image. By calibrating the predicted labels based on the pseudo-labels, the purpose of calibrating the pseudo-labels is achieved, thereby using more accurate optimized labels to train the preset student model, resulting in a more accurate target student model. This achieves the technical effect of training a target student model with higher accuracy, thus solving the technical problem of low accuracy of student models trained based on traditional pseudo-label methods. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0020] Figure 1 This is a flowchart of a model training method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of a model architecture according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of a semi-supervised learning process according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of a weak enhancement according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of a strong enhancement according to an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram illustrating the determination of a foreground frame according to an embodiment of the present invention;

[0026] Figure 7This is a schematic diagram of a foreground frame label confidence mark according to an embodiment of the present invention;

[0027] Figure 8 This is a schematic diagram of a model training device according to an embodiment of the present invention;

[0028] Figure 9 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] According to an embodiment of the present invention, a model training method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] Figure 1 This is a flowchart of a model training method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0033] Step S102: Obtain a first image set, wherein the first image set includes: multiple first images and pseudo-labels corresponding to each first image;

[0034] Step S104: Analyze multiple first images in the first image set using a preset student model to determine the predicted label corresponding to each first image. The preset student model is trained by machine learning using a second image set, which includes multiple second images and the true label corresponding to each second image.

[0035] Step S106: The predicted labels are calibrated based on the pseudo-labels to determine the optimized label corresponding to each first image;

[0036] Step S108: Train the preset student model using the third image set to obtain the target student model. The third image set includes: multiple first images and the optimized label corresponding to each first image.

[0037] In this embodiment of the invention, a first image set is obtained, comprising multiple first images and pseudo-labels corresponding to each first image; a preset student model is used to analyze the multiple first images in the first image set to determine the predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, which comprises multiple second images and true labels corresponding to each second image; the predicted labels are calibrated based on the pseudo-labels to determine the optimized label corresponding to each first image; the preset student model is trained using a third image set to obtain a target student model, wherein the third image set comprises multiple first images and optimized labels corresponding to each first image. By calibrating the predicted labels based on the pseudo-labels, the purpose of calibrating the pseudo-labels is achieved, thereby using more accurate optimized labels to train the preset student model, resulting in a more accurate target student model. This achieves the technical effect of training a target student model with higher accuracy, thus solving the technical problem of low accuracy of student models trained based on traditional pseudo-label methods.

[0038] In step S102 above, pseudo-labels can be added to the first image that has not been labeled according to the preset teacher model, so as to obtain the first image set.

[0039] Optionally, the preset teacher model and preset student model can adopt a large visual model, which can simulate the working principle of human vision, carefully analyze and process images, and continuously optimize its own algorithm through machine learning.

[0040] Optionally, the preset teacher model and the preset student model can adopt the same model architecture. Each basic module in the preset teacher model and the preset student model includes: a deformable convolutional network and a feedforward neural network. The feedforward neural network includes two fully connected networks and an activation function. The outputs of the deformable convolutional network and the feedforward neural network are respectively connected to layer normalization.

[0041] Optionally, the preset teacher model and the preset student model can be two-stage models. The first stage of the model is used to generate a series of candidate boxes based on the input image, and the second stage is used to classify the images according to the image features within the candidate boxes. For example, the preset teacher model can determine the bounding box of the first image in the first stage and add pseudo-labels according to the image features within the bounding box in the second stage.

[0042] Optionally, the pre-defined teacher model can be trained using a second set of images via machine learning, wherein the second set of images includes: multiple second images and a true label corresponding to each second image.

[0043] In step S104 above, the preset student model can be pre-trained using the second image set through machine learning. During the process of training the preset student model using the second image set, the loss function for training the model can be the cross-entropy loss function.

[0044] It should be noted that since the pseudo-labels are added to the first image without labels using a pre-set teacher model, the pseudo-labels for the first image may not be accurate enough. Therefore, in order to ensure that the target student model trained with pseudo-labels is accurate enough, the pseudo-labels for the first image need to be calibrated.

[0045] In step S106 above, the pseudo-label is determined based on the preset teacher model, and the predicted label is determined based on the preset student model. Since the pseudo-label and the predicted label are determined by different models, the analysis results of different models on the same first image may be different. In order to ensure the accuracy of the subsequent training of the target student model, the predicted label can be used to calibrate the pseudo-label to obtain a more accurate optimized label. Then, the target student model can be trained based on the optimized label to ensure that the trained target student model is accurate enough.

[0046] In step S108 above, the target student model can be jointly trained using the second image set and the third image set through machine learning. During the training of the target student model using the second image set, the loss function of the training model can be the cross-entropy loss function. During the training of the target student model using the second image set, the loss function of the training model can be the consistency loss function, which uses the KL divergence calculation formula.

[0047] It should be noted that labels can represent classification results determined based on image features in an image. Among them, true labels can represent classification results predetermined based on image features of the second image, pseudo labels are classification results predicted using a preset teacher model based on image features of the first image, and predicted labels are classification results predicted using a preset student model based on image features of the first image. Therefore, pseudo labels and predicted labels are prediction results, which may be correct or incorrect. So, in the process of determining pseudo labels and predicted labels of the first image, the label confidence of the pseudo labels or predicted labels can be determined simultaneously. The label confidence indicates the accuracy of the classification result indicated by the label.

[0048] Optionally, in the process of determining the classification result based on image features, the image may include the main object, background, and other interfering objects. The classification result of the image is for the main object of the image, but the image features of the background and other interfering objects in the image will affect the classification of the main object. Therefore, it is necessary to select the main object in the image by using candidate boxes, and then classify it according to the image features in the candidate boxes, or add labels (such as pseudo labels or predicted labels).

[0049] Optionally, the same image can have multiple labels. For example, the first image can have pseudo-labels and predicted labels. The pseudo-labels and predicted labels are classification results determined based on image features of different parts of the same image. Therefore, in order to distinguish the different image feature parts corresponding to different labels, the image feature parts for which pseudo-labels are determined can be defined as bounding boxes, and the pseudo-labels represent the classification results determined based on the image features within the bounding boxes; the image feature parts for which predicted labels are determined can be defined as candidate boxes, and the predicted labels represent the classification results determined based on the image features within the candidate boxes; the image feature parts for which optimized labels are determined can be defined as foreground boxes, and the optimized labels represent the classification results determined based on the image features within the foreground boxes.

[0050] As an optional embodiment, before obtaining the first image set, the method further includes: obtaining a sample image set, wherein the sample image set includes: multiple unlabeled first images; adding pseudo-labels to the sample image set using a preset teacher model to determine the first image set, wherein the preset teacher model is trained by machine learning using a second image set.

[0051] In the above embodiments of the present invention, a preset teacher model can be trained by using multiple second images in the second image set and the true labels corresponding to each second image as training data. Then, the trained preset teacher model is used to analyze multiple first images in the sample set that have not been labeled, and multiple first images with added pseudo labels can be obtained. The multiple first images with added pseudo labels are determined as the first image set, thus completing the addition of pseudo labels and realizing the determination of the first image set.

[0052] As an optional embodiment, adding pseudo-labels to the sample image set using a preset teacher model, and determining the first image set includes: analyzing multiple first images in the sample images using the preset teacher model, determining the pseudo-label and label confidence level corresponding to each first image; and determining the first image set based on pseudo-labels whose label confidence level is greater than a preset confidence threshold.

[0053] In the above embodiments of the present invention, since the pseudo-label added to the first image in the sample image set is a prediction result, which may be correct or incorrect, the label confidence of the pseudo-label or the predicted label can be determined simultaneously during the process of determining the pseudo-label of the first image. The label confidence indicates the accuracy of the classification result indicated by the pseudo-label. If the label confidence is not greater than a preset confidence threshold, it means that the pseudo-label is not accurate enough. Therefore, training the target student model based on the inaccurate pseudo-label will reduce the accuracy of the target student model. Therefore, pseudo-labels with label confidence not greater than the preset confidence threshold and the first image with the pseudo-label can be deleted from the sample image set. If the label confidence is greater than the preset confidence threshold, it means that the pseudo-label is accurate enough. Therefore, training the target student model based on the accurate pseudo-label will improve the accuracy of the target student model. Therefore, pseudo-labels with label confidence greater than the preset confidence threshold and the first image with the pseudo-label can be retained in the sample image set, thereby obtaining the first image set of pseudo-labels.

[0054] It should be noted that the training data used for the sample image set used to train the preset student model or preset teacher model can include sample images of multiple label categories. Different label categories represent different classification results, but sample images of different label categories may have different sample numbers. Therefore, the label category with a larger number of samples can be identified as the large sample category (such as the first category), and the label category with a smaller number of samples can be identified as the small sample category (such as the second category).

[0055] Optionally, since the number of samples in the large sample category is large, some sample images can be deleted appropriately, and only pseudo-labels with high label confidence and their sample images (i.e., the first image) can be retained; since the number of samples in the small sample category is small, as many sample images as possible need to be retained, and pseudo-labels with low label confidence and their sample images (i.e., the first image) can be retained appropriately.

[0056] Optionally, the large sample category can account for 80% of the sample image set, while the small sample category can account for 20%.

[0057] Alternatively, the sample category can be determined based on a second set of images with added true labels.

[0058] Optionally, by analyzing the true labels of each second image in the second image set, the label category indicated by each true label is determined; the label categories indicated by all true labels in the second image set are sorted; and a first category and a second category are determined according to the sorting results in a preset ratio, wherein the number of true labels in the first category (i.e., the number of second images corresponding to the true label) is greater than the number of true labels in the second category (i.e., the number of second images corresponding to the true label).

[0059] Optionally, the preset ratio can be 80:20.

[0060] As an optional embodiment, before determining the first image set based on pseudo-labels whose label confidence is greater than a preset confidence threshold, the method further includes: identifying the label category of the pseudo-labels, wherein the label category includes at least: a first category and a second category; determining the preset confidence threshold corresponding to the label category, wherein the preset confidence threshold includes at least: a first confidence threshold corresponding to the first category and a second confidence threshold corresponding to the second category, wherein the first confidence threshold and the second confidence threshold are different.

[0061] In the above embodiments of the present invention, different pseudo-labels can have different label categories. In the first image set, the number of first images of different label categories may vary. In order to ensure that each label category has samples that can train the target student model, it is necessary to set a corresponding preset confidence threshold for different label categories. Then, based on the preset confidence threshold corresponding to different label categories, pseudo-labels with higher label confidence can be retained in the label category, thereby achieving the screening of pseudo-labels of different label categories.

[0062] As an optional embodiment, determining the first image set based on pseudo-labels whose label confidence is greater than a preset confidence threshold includes: determining pseudo-labels whose label confidence is greater than a preset confidence threshold as target labels; determining the first image corresponding to the target label as the target image in the sample image set; and saving the target image and the target label corresponding to the target image to obtain the first image set.

[0063] In the above embodiments of the present invention, during the process of adding pseudo-labels to the first image in the sample image set, pseudo-labels with high label confidence and the first image corresponding to the pseudo-labels can be filtered according to a preset confidence threshold, and the filtered pseudo-labels and the first images are used as the first image set to determine the first image set.

[0064] As an optional embodiment, before adding pseudo-labels to the sample image set using a preset teacher model and determining the first image set, the method further includes: weakly enhancing the first image in the sample image set, wherein the weak enhancement includes at least one of the following: randomly flipping the second image, randomly scaling the second image, and randomly rotating the second image.

[0065] In the above embodiments of the present invention, by performing weak enhancement on the first image in the sample image set, a small degree of transformation can be made to the first image. The environment usually does not change the basic features of the first image, thus achieving a small perturbation to the first image, enabling the preset teacher model to learn more local features, thereby improving the performance of the preset teacher model.

[0066] As an optional embodiment, before analyzing multiple first images in the first image set using a preset student model to determine the predicted label corresponding to each first image, the method further includes: performing weak enhancement on the first images in the first image set, wherein the weak enhancement is at least one of the following: random flipping, random scaling transformation, and random rotation; and performing strong enhancement on the first images in the first image set, wherein the strong enhancement is at least one of the following: blending and image matting.

[0067] In the above embodiments of the present invention, weak enhancement can transform the first image to a small extent without changing its basic features, thereby improving the learning ability of the preset student model for local features; strong enhancement can transform the first image to a large extent, changing some of its basic features, thereby improving the robustness of the preset student model.

[0068] As an optional embodiment, calibrating the predicted label based on the pseudo-label and determining the optimized label corresponding to each first image includes: determining the pseudo-label overlap rate between the pseudo-label and the predicted label, wherein the pseudo-label is associated with the bounding box in the first image, the predicted label is associated with the candidate box in the first image, and the pseudo-label overlap rate is the intersection-union ratio of the bounding box and the candidate box; determining the foreground box of the first image based on the pseudo-label overlap rate; and determining the predicted label associated with the foreground box based on the pseudo-label.

[0069] In the above embodiments of the present invention, the pseudo-label is associated with the bounding box in the first image, and the predicted label is associated with the candidate box in the first image. Since the bounding box is determined based on a preset teacher model and the candidate box is determined based on a preset student model, the bounding box and the candidate box may be different. By calculating the intersection-union ratio of the bounding box and the candidate box, the area overlap of the bounding box and the candidate box can be determined. Then, based on the bounding box and the candidate box, the foreground box can be re-determined in the first image, and the classification result indicated by the pseudo-label based on the image features within the bounding box is determined as the classification result determined by the predicted label based on the image features within the foreground box, thereby achieving the calibration of the pseudo-label.

[0070] Optionally, the predicted label associated with the foreground box based on the pseudo label includes: determining the classification result indicated by the pseudo label based on the image features within the labeled box as the classification result determined by the predicted label based on the image features within the foreground box.

[0071] Optionally, if multiple candidate boxes are determined based on a preset student model, the intersection-union ratio (IUR) between each candidate box and the labeled box can be calculated, and the candidate box with the highest IUR is retained.

[0072] Optionally, candidate boxes with an intersection-union ratio (IU) greater than a preset IU threshold can be retained; or candidate boxes with the largest IU ratio can be retained.

[0073] As an optional embodiment, determining the foreground box of the first image based on the pseudo-label overlap rate includes: if the pseudo-label overlap rate is greater than a preset overlap threshold, determining the middle box between the label box and the candidate box as the foreground box, wherein the middle box is determined based on the median coordinate value of the label box and the candidate box; or if the pseudo-label overlap rate is not greater than the preset overlap threshold, determining the label box as the foreground box.

[0074] In the above embodiments of the present invention, when the pseudo-label overlap rate is greater than a preset overlap threshold, that is, when the intersection-union ratio of the labeled box and the candidate box is greater than the preset overlap threshold, that is, when there is a large area of ​​overlap between the labeled box and the candidate box, it can be said that the pseudo-label and the predicted label are similar, and it can also be said that the pseudo-label and the predicted label are relatively close to the true label. In order to further improve the accuracy of the pseudo-label, the foreground box and the optimized label can be determined based on the median coordinate value of the labeled box and the candidate box in the first image, so that the optimized label can be closer to the true label.

[0075] In the above embodiments of the present invention, when the pseudo-label overlap rate is not greater than a preset overlap threshold, that is, when the intersection-union ratio of the labeled box and the candidate box is not greater than the preset overlap threshold, that is, when the overlapping area of ​​the labeled box and the candidate box is small, it can be said that the pseudo-label and the predicted label are significantly different. Since the preset teacher model is more accurate than the preset student model, the pseudo-label is closer to the true label and can be directly used as the optimized label.

[0076] As an optional embodiment, after predicting the label based on the pseudo-label and the foreground box, the method further includes: determining the label confidence of the pseudo-label relative to the label box as the label confidence of the predicted label relative to the foreground box.

[0077] In the above embodiments of the present invention, after determining the foreground box, the label confidence of the pseudo label relative to the image features within the labeled box can be determined as the label confidence of the predicted label relative to the image features within the foreground box, thereby realizing the determination of the label confidence of the predicted label.

[0078] This invention also provides an optional embodiment, which offers a semi-supervised learning method for object detection in scenarios with small and long-tailed samples. This method aims to optimize the problem of noisy pseudo-labels from several perspectives: First, a large visual model is used as the teacher model. Compared to commonly used visual models, the general technical capabilities of the large model make it better at handling long-tailed learning problems. Therefore, when transferring to new scenarios, the large model typically performs better in classification and labeling tasks, thus improving the quality of pseudo-labels. Second, a pseudo-label calibration process is used to optimize pseudo-labels. Third, by introducing a consistency loss function during the training of the student model, a self-supervised learning process for the student model is achieved, significantly improving the student model's ability to learn from unlabeled samples. Finally, to further reduce the negative impact of low-confidence pseudo-labels, a backpropagation gradient control method based on pseudo-label confidence is used. By using the confidence of the pseudo-label as the weight of the backpropagation gradient, the teacher model can significantly reduce the negative impact of erroneous pseudo-labels on the student model, solving the problem of low student model accuracy and practical difficulty in practical application of traditional semi-supervised learning methods in scenarios with small and long-tailed samples.

[0079] Optionally, the technical framework adopted in this application can be used to train commonly used asynchronous detection algorithms such as Faster R-CNN, realizing the practical application of semi-supervised learning algorithms.

[0080] As an optional embodiment, the semi-supervised learning method for object detection provided in this application includes:

[0081] Step 1: Select 10% of the data from the entire unannotated training dataset and manually annotate the dataset. The annotated dataset (e.g., the second image set) is defined as [X1, Y1]. L Unlabeled datasets (such as a set of sample images or a first set of images) are defined as [X U ,*] U Let the test set be [X]. T ,*] T .

[0082] Step 2: Using the labeled dataset (such as the second image set) as [X1, Y1] L The pre-set teacher model is trained, and the trained pre-set teacher model is denoted as follows:

[0083] Optionally, count the number of instances (i.e., the number of first images) for all label categories in the labeled dataset (such as the second image set), and calculate the instance proportion R for each label category. C Then, sort all tag categories according to the proportion of instances.

[0084] Optionally, according to the 80 / 20 rule, the top 80% are considered large sample categories, and the rest are considered small sample categories.

[0085] Optionally, The total number of samples for all categories can be the number of second images in the second image set, and the total number of samples for each category can be the number of second images for each label category.

[0086] Figure 2 This is a schematic diagram of a model architecture according to an embodiment of the present invention, such as... Figure 2As shown, the pre-defined teacher and student models employ deformable convolutional networks (DCNs) instead of the convolutional networks (CNNs) or multi-head self-attention networks (MHSAs) commonly used in basic vision models. Building upon DCNv2, the pre-defined teacher and student models replace traditional convolutions with depth-wise convolutions and point-wise convolutions, and perform grouped learning of parameters. This gives DCN both the inductive bias capability of CNNs and the global learning capability of MHSAs. In constructing the basic network units, the ViT transformer architecture is adopted. Each basic unit contains a DCN, layer normalization, and a feed-forward neural network (FFN). The FFN includes two fully connected networks (FCs) and an activation function ReLU. Through these modifications, the pre-defined teacher and student models achieve state-of-the-art (SOTA) performance on multiple public object detection datasets such as COCO, VOC, and Cityscapes.

[0087] It's important to note that SOTA models refer to "State-of-the-Art" models, also known as state-of-the-art models. They are models or algorithms that are currently considered the best in a specific task or domain. SOTA models are typically trained and optimized on large-scale datasets, utilizing state-of-the-art techniques and methods to achieve optimal performance. These models are generally widely accepted and applied in academia and industry because they exhibit the highest accuracy, efficiency, or other metrics on specific tasks. The emergence of SOTA models often represents the latest research progress in the field.

[0088] Optionally, when fine-tuning the base model of the preset teacher or student model using a small dataset, considering the complexity of the DCN and the limitation of data volume, the DCN network in the backbone network is fixed, while the FC layer in the FFN is fine-tuned with a learning rate of 0.001. This not only avoids the backbone network from overfitting to a small amount of data, but also allows the network to focus more on important elements in the image. In addition, the head network of the model is retrained with a learning rate of 0.01. Considering that the network architecture of the preset teacher and student models is similar to that of traditional visual detection models, the fine-tuning method is also similar to commonly used fine-tuning techniques.

[0089] Optionally, AdamW can be selected as the optimization strategy during fine-tuning training, with a decay strategy of lr**epoch, where lr is the learning rate and epoch is the number of training epochs. This allows the model to converge faster.

[0090] Alternatively, when the model accuracy stabilizes at a certain value, SGD can be used instead of AdamW. By fixing all backbone networks and fine-tuning the head network with a learning rate of 0.01, the network accuracy can be improved slightly.

[0091] Step 3: Utilize the labeled dataset [X1, Y1] L (i.e., the second image set) and the unlabeled dataset [X] U ,*] U (e.g., a set of sample images or a first set of images) Train a pre-defined student model MS, and finally obtain the target student model.

[0092] Figure 3 This is a schematic diagram of a semi-supervised learning process according to an embodiment of the present invention, such as... Figure 3 As shown, there are 4 branches, of which branch 1 is a supervised learning branch; branches 2-4 are unsupervised learning branches.

[0093] Optionally, branch 1 trains a pre-defined student model using a labeled dataset (i.e., the second set of images), and the training process uses the cross-entropy loss function CE Loss.

[0094] Optionally, branches 2 and 3 train a preset student model using an unlabeled dataset (i.e., the second set of images), and the training process uses the Consistency Loss function.

[0095] Optionally, the preset teacher model can be frozen during training, while the preset student model can be trained normally.

[0096] Optionally, each training round reads 4*GPU number of images, and each GPU processes 4 images in parallel, including 2 labeled images (such as the second image) and 2 unlabeled images (such as the first image).

[0097] Optionally, branch 1 can perform strong enhancement on the labeled image (i.e., the second image) and then train the preset student model.

[0098] Figure 4 This is a schematic diagram of a weak enhancement according to an embodiment of the present invention, such as... Figure 4 As shown, weak enhancement includes three functions: random flipping, random scaling, and random rotation.

[0099] Figure 5 This is a schematic diagram of a strongly enhanced embodiment of the present invention, such as... Figure 5 As shown, strong enhancement is based on weak enhancement and performs MixUp and CutOut operations.

[0100] Optionally, branch 4 is used to add pseudo-labels to the first image.

[0101] Optionally, branch 4 enhances the first image using a weak enhancement method, and then passes it through a pre-trained preset teacher model. Get the pseudo-label set P G (i.e., the first set of images), where the position of each pseudo-label is P (i.e., the position of the pseudo-label's bounding box in the first image), the classification result of each pseudo-label is C, and the label confidence of each pseudo-label is β.

[0102] Optionally, the pseudo-label set P G (i.e., the first image set) only retains pseudo-labels for large sample categories with label confidence above 80% and pseudo-labels for small sample categories with label confidence above 50%.

[0103] Optionally, branch 2 is used to perform image enhancement on the first image.

[0104] Optionally, branch 2 enhances the unlabeled image (i.e., the first image) using a weak enhancement method, and then predicts candidate boxes using a preset student model.

[0105] Optionally, branch 3 enhances the unlabeled image (i.e., the first image) using a strong enhancement method, and then predicts foreground candidate boxes using a preset student model.

[0106] Figure 6 This is a schematic diagram of determining a foreground frame according to an embodiment of the present invention, such as... Figure 6 As shown, branch 2 and branch 3 can generate the foreground box of the first image. The specific implementation method is as follows:

[0107] The RPN network for the student model generates candidate boxes P. O i ;

[0108] calculate

[0109] Wherein, the candidate box is P O i Let i be the i-th candidate box, and let P be the pseudo-label with a confidence score greater than 0.8. G i Optimize the tag to P F k k is the kth optimized label, and only the candidate box with the highest overlap rate with the pseudo label generated by branch 4 is selected as the foreground box.

[0110] Figure 7 This is a schematic diagram of a foreground frame label confidence marker according to an embodiment of the present invention, such as... Figure 7 As shown, branch 2 and branch 3 are used to label the confidence level of the foreground box.

[0111] For example, a foreground box that highly overlaps with a pseudo-label with a label confidence of 0.9 has a label confidence of 0.9; a foreground box that highly overlaps with a pseudo-label with a label confidence of 0.4 has a label confidence of 0.4.

[0112] Optionally, the foreground box determination of branch 2 and branch 3 can identify candidate boxes with an overlap ratio (IoU) greater than 0.6 with the pseudo-label as foreground boxes.

[0113] Optionally, the loss function for training the preset student model in branch 1 is the commonly used cross-entropy loss function CE Loss.

[0114] Optionally, the loss function of the preset student model for branch 2 and branch 3 is the consistency loss function, which adopts the KL divergence calculation formula based on the weakly enhanced branch.

[0115] Optionally, the KL divergence calculation results of each foreground box in branch 2 and branch 3 are multiplied by the label confidence of each box, thereby suppressing the influence of foreground boxes with low label confidence on the model.

[0116] Step 4: After training, only branch 1 and the target student model are retained.

[0117] Step 5: Utilize the test set [X] T ,*] T Test target student model

[0118] As an example, in an industrial visual quality inspection project, hundreds of thousands of data points were collected during the sample collection period. The relevant detection targets included defects such as scratches, bumps, and dents. Approximately 20% of the defect images were randomly selected. Among these, dents and bumps accounted for over 6,000 samples, while scratches accounted for only 260. The Faster R-CNN visual detection model, trained on an imbalanced training dataset, achieved approximately 80% accuracy in detecting large-sample defects such as bumps and dents, and 50% accuracy in detecting small-sample defects such as scratches.

[0119] To address the aforementioned issue, 77% of the large visual model (i.e., the target student model) was trained using the same imbalanced samples. The results showed that the large visual model achieved approximately 82% accuracy in detecting large sample categories, but only 63% accuracy in detecting small samples. Particularly for 'scratches', the detection accuracy improved from 55% to 77%, essentially achieving detection capability. These experimental results suggest that in a semi-supervised learning system, the large model is more suitable as the initial teacher model.

[0120] Finally, the aforementioned semi-supervised learning strategy was introduced to simultaneously train the Faster R-CNN detection model (i.e., the target student model) using labeled data (such as the second image with a true label) and unlabeled data (such as the first image without a label, or the first image with a pseudo label). Ultimately, an accuracy of 88% was achieved for large-sample defect detection and 80% for small-sample defect detection, demonstrating the feasibility of the algorithm.

[0121] The technical solution provided in this application improves the quality of pseudo-labels by using a large visual model, and reduces the negative impact of erroneous pseudo-labels on training student models by using pseudo-label accuracy correction, self-supervised learning, and backpropagation gradient control based on pseudo-label confidence. This overcomes the problems of low student model accuracy and difficulty in practical application of traditional semi-supervised learning methods in scenarios with small samples and long-tailed samples.

[0122] This application uses a large visual model to optimize pseudo-labels. Compared with commonly used visual models, the general technical capabilities of the large model make it better at handling long-tailed samples and small samples, and the quality of the pseudo-labels it provides is significantly better than other algorithms. Gradient backpropagation is controlled by the label confidence of the pseudo-labels, which further reduces the negative impact of noisy pseudo-labels on student model training. A relatively simple network architecture is adopted, which has certain practical application capabilities and combines innovation and practicality.

[0123] According to an embodiment of the present invention, a model training device embodiment is also provided. It should be noted that the model training device can be used to execute the model training method in the embodiment of the present invention, and the model training method in the embodiment of the present invention can be executed in the model training device.

[0124] Figure 8 This is a schematic diagram of a model training device according to an embodiment of the present invention, such as... Figure 8 As shown, the device may include: an acquisition module 82, used to acquire a first image set, wherein the first image set includes: multiple first images and pseudo-labels corresponding to each first image; an analysis module 84, used to analyze the multiple first images in the first image set using a preset student model to determine the predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, the second image set including: multiple second images and true labels corresponding to each second image; a determination module 86, used to calibrate the predicted labels based on the pseudo-labels to determine the optimized label corresponding to each first image; and a training module 88, used to train the preset student model using a third image set to obtain a target student model, wherein the third image set includes: multiple first images and optimized labels corresponding to each first image.

[0125] It should be noted that the acquisition module 82 in this embodiment can be used to execute step S102 in this application embodiment, the analysis module 84 in this embodiment can be used to execute step S104 in this application embodiment, the determination module 86 in this embodiment can be used to execute step S106 in this application embodiment, and the training module 88 in this embodiment can be used to execute step S108 in this application embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments.

[0126] In this embodiment of the invention, a first image set is obtained, comprising multiple first images and pseudo-labels corresponding to each first image; a preset student model is used to analyze the multiple first images in the first image set to determine the predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, which comprises multiple second images and true labels corresponding to each second image; the predicted labels are calibrated based on the pseudo-labels to determine the optimized label corresponding to each first image; the preset student model is trained using a third image set to obtain a target student model, wherein the third image set comprises multiple first images and optimized labels corresponding to each first image. By calibrating the predicted labels based on the pseudo-labels, the purpose of calibrating the pseudo-labels is achieved, thereby using more accurate optimized labels to train the preset student model, resulting in a more accurate target student model. This achieves the technical effect of training a target student model with higher accuracy, thus solving the technical problem of low accuracy of student models trained based on traditional pseudo-label methods.

[0127] As an optional embodiment, the apparatus further includes: an acquisition submodule, configured to acquire a sample image set before acquiring the first image set, wherein the sample image set includes: a plurality of first images without labels; and a determination submodule, configured to add pseudo-labels to the sample image set using a preset teacher model, and determine the first image set, wherein the preset teacher model is trained by machine learning using a second image set.

[0128] As an optional embodiment, the determination submodule includes: a first determination unit, used to analyze multiple first images in the sample images using a preset teacher model, and determine the pseudo label and label confidence level corresponding to each first image; and a second determination unit, used to determine the set of first images based on pseudo labels whose label confidence level is greater than a preset confidence threshold.

[0129] As an optional embodiment, the apparatus further includes: an identification unit, configured to identify the label category of a pseudo-label before determining the first image set based on pseudo-labels whose label confidence is greater than a preset confidence threshold, wherein the label category includes at least: a first category and a second category; and a third determination unit, configured to determine the preset confidence threshold corresponding to the label category, wherein the preset confidence threshold includes at least: a first confidence threshold corresponding to the first category and a second confidence threshold corresponding to the second category, wherein the first confidence threshold and the second confidence threshold are different.

[0130] As an optional embodiment, the third determining unit includes: a first determining subunit, used to determine pseudo-labels with a label confidence level greater than a preset confidence threshold as target labels; a second determining subunit, used to determine the first image corresponding to the target label as the target image in the sample image set; and a saving subunit, used to save the target image and the target label corresponding to the target image to obtain the first image set.

[0131] As an optional embodiment, the apparatus further includes: a first processing unit, configured to add pseudo-labels to the sample image set using a preset teacher model, and before determining the first image set, perform weak enhancement on a first image in the sample image set, wherein the weak enhancement includes at least one of the following: randomly flipping the second image, randomly scaling the second image, and randomly rotating the second image.

[0132] As an optional embodiment, the apparatus further includes: a second processing unit, configured to perform weak enhancement on the first images in the first image set before analyzing multiple first images in the first image set using a preset student model and determining the prediction label corresponding to each first image, wherein the weak enhancement includes at least one of the following: random flipping, random scaling transformation, and random rotation; and a third processing unit, configured to perform strong enhancement on the first images in the first image set, wherein the strong enhancement includes at least one of the following: blending and image matting.

[0133] As an optional embodiment, the determining module includes: a fourth determining unit, used to determine the pseudo-label overlap rate of the pseudo-label and the predicted label, wherein the pseudo-label is associated with the bounding box in the first image, the predicted label is associated with the candidate box in the first image, and the pseudo-label overlap rate is the intersection-union ratio of the bounding box and the candidate box; a fifth determining unit, used to determine the foreground box of the first image based on the pseudo-label overlap rate; and a sixth determining unit, used to determine the predicted label associated with the foreground box based on the pseudo-label.

[0134] As an optional embodiment, the fifth determining unit includes: a seventh determining unit, used to determine the middle box of the label box and the candidate box as the foreground box when the pseudo-label overlap rate is greater than a preset overlap threshold, wherein the middle box is determined according to the median coordinate value of the label box and the candidate box; or an eighth determining unit, used to determine the label box as the foreground box when the pseudo-label overlap rate is not greater than a preset overlap threshold.

[0135] As an optional embodiment, the sixth determining unit includes: a ninth determining unit, used to determine the label confidence of the pseudo label relative to the label box as the label confidence of the predicted label relative to the foreground box.

[0136] Embodiments of the present invention can provide a computer terminal, which can be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the computer terminal can also be replaced by a mobile terminal or other terminal device.

[0137] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0138] In this embodiment, the computer terminal described above can execute the program code for the following steps in the model training method: obtaining a first image set, wherein the first image set includes: multiple first images and pseudo-labels corresponding to each first image; analyzing the multiple first images in the first image set using a preset student model to determine the predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, the second image set including: multiple second images and true labels corresponding to each second image; calibrating the predicted labels based on the pseudo-labels to determine the optimized label corresponding to each first image; training the preset student model using a third image set to obtain a target student model, wherein the third image set includes: multiple first images and optimized labels corresponding to each first image.

[0139] Optionally, Figure 9 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 9 As shown, the computer terminal 90 may include one or more (only one is shown in the figure) processors 92 and memory 94.

[0140] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the model training method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned model training method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 90 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0141] The processor can invoke information and application programs stored in the memory via a transmission device to perform the following steps: acquiring a first image set, wherein the first image set includes: multiple first images and pseudo-labels corresponding to each first image; analyzing the multiple first images in the first image set using a preset student model to determine the predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, the second image set including: multiple second images and true labels corresponding to each second image; calibrating the predicted labels based on the pseudo-labels to determine the optimized label corresponding to each first image; training the preset student model using a third image set to obtain a target student model, wherein the third image set includes: multiple first images and optimized labels corresponding to each first image.

[0142] Optionally, the processor may also execute program code for the following steps: before obtaining the first image set, obtaining a sample image set, wherein the sample image set includes: multiple first images without labels; adding pseudo-labels to the sample image set using a preset teacher model, and determining the first image set, wherein the preset teacher model is trained by machine learning using a second image set.

[0143] Optionally, the processor may also execute program code for the following steps: using a preset teacher model to analyze multiple first images in the sample images, determining the pseudo-label and label confidence level corresponding to each first image; and determining the set of first images based on pseudo-labels whose label confidence level is greater than a preset confidence threshold.

[0144] Optionally, the processor may also execute program code for the following steps: before determining the first image set based on pseudo-labels whose label confidence is greater than a preset confidence threshold, identifying the label category of the pseudo-labels, wherein the label category includes at least: a first category and a second category; determining the preset confidence threshold corresponding to the label category, wherein the preset confidence threshold includes at least: a first confidence threshold corresponding to the first category and a second confidence threshold corresponding to the second category, wherein the first confidence threshold and the second confidence threshold are different.

[0145] Optionally, the processor may also execute program code that performs the following steps: determining pseudo-labels whose label confidence is greater than a preset confidence threshold as target labels;

[0146] In the sample image set, the first image corresponding to the target label is determined as the target image; the target image and the target label corresponding to the target image are saved to obtain the first image set.

[0147] Optionally, the processor may also execute program code that performs the following steps: before adding pseudo-labels to the sample image set using a preset teacher model and determining the first image set, weakly enhances the first image in the sample image set, wherein the weak enhancement includes at least one of the following: randomly flipping the second image, randomly scaling the second image, and randomly rotating the second image.

[0148] Optionally, the processor may also execute program code for the following steps: before analyzing multiple first images in the first image set using a preset student model and determining the predicted label corresponding to each first image, weakly enhancing the first images in the first image set, wherein the weak enhancement is at least one of the following: random flipping, random scaling transformation, and random rotation; and strongly enhancing the first images in the first image set, wherein the strong enhancement is at least one of the following: blending and image matting.

[0149] Optionally, the processor may also execute program code for the following steps: determining the pseudo-label overlap rate of pseudo-labels and predicted labels, wherein the pseudo-labels are associated with bounding boxes in the first image, the predicted labels are associated with candidate boxes in the first image, and the pseudo-label overlap rate is the intersection-union ratio of the bounding boxes and candidate boxes; determining the foreground box of the first image based on the pseudo-label overlap rate; and determining the predicted label associated with the foreground box based on the pseudo-labels.

[0150] Optionally, the processor may also execute program code that performs the following steps: when the pseudo-label overlap rate is greater than a preset overlap threshold, the middle box between the label box and the candidate box is determined as the foreground box, wherein the middle box is determined based on the midpoint of the coordinates of the label box and the candidate box; or when the pseudo-label overlap rate is not greater than a preset overlap threshold, the label box is determined as the foreground box.

[0151] Optionally, the processor may also execute program code that performs the following steps: determining the label confidence of the pseudo-label relative to the bounding box as the label confidence of the predicted label relative to the foreground box.

[0152] This invention provides a model training scheme. In this embodiment, a first image set is obtained, comprising multiple first images and pseudo-labels corresponding to each first image. A preset student model is used to analyze the multiple first images in the first image set to determine a predicted label for each first image. The preset student model is trained using a second image set through machine learning. The second image set includes multiple second images and true labels corresponding to each second image. The predicted labels are calibrated based on the pseudo-labels to determine an optimized label for each first image. A third image set is used to train the preset student model to obtain a target student model. The third image set includes multiple first images and optimized labels corresponding to each first image. By calibrating the predicted labels based on the pseudo-labels, the purpose of calibrating the pseudo-labels is achieved. This allows for training the preset student model with more accurate optimized labels, resulting in a more accurate target student model. This achieves the technical effect of training a target student model with higher accuracy, thereby solving the technical problem of low accuracy in student models trained based on traditional pseudo-label methods.

[0153] Those skilled in the art will understand that Figure 9 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 9 This does not limit the structure of the aforementioned electronic device. For example, the computer terminal 90 may also include components that are more... Figure 9 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 9 The different configurations shown.

[0154] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a non-volatile medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0155] Embodiments of the present invention also provide a non-volatile storage medium. Optionally, in this embodiment, the aforementioned non-volatile storage medium can be used to store the program code executed by the model training method provided in the above embodiments.

[0156] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0157] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining a first image set, wherein the first image set includes: a plurality of first images and pseudo-labels corresponding to each first image; analyzing the plurality of first images in the first image set using a preset student model to determine the predicted label corresponding to each first image, wherein the preset student model is trained by machine learning using a second image set, the second image set including: a plurality of second images and true labels corresponding to each second image; calibrating the predicted labels based on the pseudo-labels to determine the optimized label corresponding to each first image; training the preset student model using a third image set to obtain a target student model, wherein the third image set includes: a plurality of first images and optimized labels corresponding to each first image.

[0158] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: before obtaining the first image set, obtaining a sample image set, wherein the sample image set includes: multiple first images without labels; adding pseudo-labels to the sample image set using a preset teacher model, and determining the first image set, wherein the preset teacher model is trained by machine learning using a second image set.

[0159] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: analyzing multiple first images in the sample images using a preset teacher model to determine the pseudo-label and label confidence level corresponding to each first image; and determining the set of first images based on pseudo-labels whose label confidence level is greater than a preset confidence threshold.

[0160] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: before determining the first image set based on pseudo-labels whose label confidence is greater than a preset confidence threshold, identifying the label category of the pseudo-labels, wherein the label category includes at least: a first category and a second category; determining the preset confidence threshold corresponding to the label category, wherein the preset confidence threshold includes at least: a first confidence threshold corresponding to the first category and a second confidence threshold corresponding to the second category, wherein the first confidence threshold and the second confidence threshold are different.

[0161] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining pseudo-labels whose label confidence is greater than a preset confidence threshold as target labels; determining the first image corresponding to the target label as the target image in the sample image set; saving the target image and the target label corresponding to the target image to obtain the first image set.

[0162] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: before adding pseudo-labels to the sample image set using a preset teacher model and determining the first image set, weakly enhancing the first image in the sample image set, wherein the weak enhancement includes at least one of the following: randomly flipping the second image, randomly scaling the second image, and randomly rotating the second image.

[0163] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: before analyzing multiple first images in the first image set using a preset student model and determining the predicted label corresponding to each first image, weakly enhancing the first images in the first image set, wherein the weak enhancement includes at least one of the following: random flipping, random scaling transformation, and random rotation; and strongly enhancing the first images in the first image set, wherein the strong enhancement includes at least one of the following: blending and image matting.

[0164] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining the pseudo-label overlap rate of pseudo-labels and predicted labels, wherein the pseudo-labels are associated with bounding boxes in the first image, the predicted labels are associated with candidate boxes in the first image, and the pseudo-label overlap rate is the intersection-union ratio of the bounding boxes and candidate boxes; determining the foreground box of the first image based on the pseudo-label overlap rate; and determining the predicted label associated with the foreground box based on the pseudo-labels.

[0165] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: when the pseudo-label overlap rate is greater than a preset overlap threshold, the middle box of the label box and the candidate box is determined as the foreground box, wherein the middle box is determined based on the midpoint of the coordinates of the label box and the candidate box; or when the pseudo-label overlap rate is not greater than the preset overlap threshold, the label box is determined as the foreground box.

[0166] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining the label confidence of the pseudo-label relative to the label box as the label confidence of the predicted label relative to the foreground box.

[0167] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0168] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0172] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned non-volatile storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0173] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A model training method, characterized in that, include: Obtain a first image set, wherein the first image set includes: a plurality of first images, and a pseudo-label corresponding to each first image; The preset student model is used to analyze multiple first images in the first image set to determine the predicted label corresponding to each first image. The preset student model is trained by machine learning using a second image set, which includes multiple second images and a true label corresponding to each second image. The pseudo-labels are calibrated based on the predicted labels to determine the optimized label for each of the first images; The preset student model is trained using a third image set to obtain a target student model, wherein the third image set includes: multiple first images and the optimized label corresponding to each first image; The process of calibrating the pseudo-labels based on the predicted labels to determine the optimized label for each of the first images includes: Determine the pseudo-label overlap rate between the pseudo-label and the predicted label, wherein the pseudo-label is associated with the bounding box in the first image, the predicted label is associated with the candidate box in the first image, and the pseudo-label overlap rate is the intersection-union ratio of the bounding box and the candidate box; If the pseudo-label overlap rate is greater than a preset overlap threshold, the foreground box and the optimized label are determined based on the median coordinates of the labeled box and the candidate box in the first image, wherein the optimized label represents the classification result determined based on the image features within the foreground box; If the pseudo-label overlap rate is not greater than a preset overlap threshold, the pseudo-label is used as the optimized label.

2. The method of claim 1, wherein, Before acquiring the first image set, the method further includes: Obtain a sample image set, wherein the sample image set includes: multiple first images without labels; The first image set is determined by adding pseudo-labels to the sample image set using a preset teacher model, wherein the preset teacher model is trained by machine learning using the second image set.

3. The method according to claim 2, characterized in that, Using a preset teacher model, pseudo-labels are added to the sample image set to determine that the first image set includes: The preset teacher model is used to analyze multiple first images in the sample images to determine the pseudo label and label confidence level corresponding to each first image; The first image set is determined based on the pseudo-labels whose label confidence is greater than a preset confidence threshold.

4. The method according to claim 3, characterized in that, Before determining the first image set based on pseudo-labels whose label confidence scores are greater than a preset confidence threshold, the method further includes: Identify the label category of the pseudo-label, wherein the label category includes at least: a first category and a second category; The preset confidence threshold corresponding to the label category is determined, wherein the preset confidence threshold includes at least: a first confidence threshold corresponding to the first category and a second confidence threshold corresponding to the second category, wherein the first confidence threshold and the second confidence threshold are different.

5. The method according to claim 1, characterized in that, The method further includes: After determining the foreground box, the label confidence of the pseudo label relative to the label box is determined as the label confidence of the optimized label relative to the foreground box.

6. A model training device, characterized in that, include: The acquisition module is used to acquire a first image set, wherein the first image set includes: a plurality of first images and a pseudo-label corresponding to each first image; The analysis module is used to analyze multiple first images in the first image set using a preset student model to determine the predicted label corresponding to each first image. The preset student model is trained by machine learning using a second image set, which includes multiple second images and a true label corresponding to each second image. The determination module is used to calibrate the pseudo-labels based on the predicted labels and determine the optimized label corresponding to each of the first images; The training module is used to train the preset student model using a third image set to obtain a target student model, wherein the third image set includes: a plurality of first images and the optimized label corresponding to each first image; The process of calibrating the pseudo-labels based on the predicted labels to determine the optimized label for each of the first images includes: Determine the pseudo-label overlap rate between the pseudo-label and the predicted label, wherein the pseudo-label is associated with the bounding box in the first image, the predicted label is associated with the candidate box in the first image, and the pseudo-label overlap rate is the intersection-union ratio of the bounding box and the candidate box; If the pseudo-label overlap rate is greater than a preset overlap threshold, the foreground box and the optimized label are determined based on the median coordinates of the labeled box and the candidate box in the first image, wherein the optimized label represents the classification result determined based on the image features within the foreground box; If the pseudo-label overlap rate is not greater than a preset overlap threshold, the pseudo-label is used as the optimized label.

7. A non-volatile storage medium, characterized in that, The non-volatile storage medium is used to store a program, wherein, when the program is running, the device where the non-volatile storage medium is located is controlled to execute the model training method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the processor, wherein the program, when running, executes the model training method according to any one of claims 1 to 5.

Images