Semi-supervised target detection model training method and device, equipment and medium

Abstract

This application provides a semi-supervised object detection model training method, apparatus, device, and medium. The method includes: acquiring a labeled dataset and an unlabeled dataset; training a teacher model with randomized weight coefficients using the labeled dataset to obtain a teacher model with updated weight coefficients; using the teacher model with updated weight coefficients to predict target objects in the unlabeled dataset; jointly training a student model with randomized weight coefficients based on target coordinates, target category, pseudo-target coordinates, and pseudo-target category to obtain a student model with updated weight coefficients; updating the weight coefficients of the teacher model with updated weight coefficients based on the student model with updated weight coefficients; and when the weight coefficients between the two meet a preset condition, using the student model with updated weight coefficients as the semi-supervised object detection model. This method splits the detection task into two sub-tasks, localization and classification, to train the model, resulting in an accurate and highly generalizable semi-supervised object detection model.

Description

Technical Field

[0001] This invention relates to the field of semi-supervised target detection technology, and in particular to a method, apparatus, equipment and medium for training a semi-supervised target detection model. Background Technology

[0002] With the development of computer technology and the application of computer vision principles, object detection has a wide range of applications, such as autonomous driving, industrial production, video surveillance, medical image processing, and satellite image processing. Object detection is mostly based on deep learning algorithms. The results depend on the labeled data used in model training, such as the object's location and category. Since training deep neural networks requires a large amount of data, and labeling training images is time-consuming and labor-intensive, semi-supervised object detection methods have emerged. Semi-supervised methods utilize both labeled and unlabeled data simultaneously—obtaining strong supervisory learning signals from labeled data and useful learning information from unlabeled training data—further improving the performance of object detection models.

[0003] Semi-supervised object detection techniques can be divided into consistency learning-based and pseudo-label learning-based algorithms. Consistency learning-based semi-supervised object detection methods identify consistent features in the same unlabeled image under perturbation, thereby utilizing the data information of the unlabeled image for object detection. Pseudo-label learning-based semi-supervised object detection algorithms learn from labeled data to obtain a pre-trained model (teacher network), then predict unlabeled data to obtain pseudo-label data, and finally retrain the model using both the pseudo-labeled and labeled data, achieving object detection through the resulting student network. However, in existing technologies, pseudo-label learning-based semi-supervised object detection does not separate the object detection task into object localization and object classification, and both the teacher and student models only use the object detection network. This results in a low ability of the semi-supervised object detection model to classify unlabeled data, leading to low accuracy in object detection results. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the present invention provides a semi-supervised object detection model training method, apparatus, device and medium to solve the technical problem of low accuracy of object detection results of semi-supervised object detection models.

[0005] To address the above problems, this invention provides a semi-supervised object detection model training method, the method comprising:

[0006] Obtain labeled and unlabeled datasets, wherein the image samples in the labeled dataset carry the target coordinates and target category of the target object;

[0007] Construct a teacher model and a student model with randomized weight coefficients, both containing the same object detection network and image classification network. Train the teacher model with randomized weight coefficients using the labeled dataset to obtain a teacher model with updated weight coefficients.

[0008] The teacher model updated with the weight coefficients is used to predict target objects in the unlabeled dataset to obtain pseudo-target coordinates and pseudo-target categories;

[0009] The student model with randomized weight coefficients is jointly trained based on the target coordinates, the target category, the pseudo-target coordinates, and the pseudo-target category to obtain a student model with updated weight coefficients.

[0010] Based on the updated weight coefficients of the student model, the updated weight coefficients of the teacher model are updated until the weight coefficients of the two models meet a preset condition. The student model with the updated weight coefficients is then used as a semi-supervised object detection model.

[0011] In one embodiment of the present invention, the teacher model with randomized weight coefficients is trained using the labeled dataset to obtain a teacher model with updated weight coefficients, including:

[0012] The target detection network in the teacher model with randomized weight coefficients is trained using the target coordinates and the target category; and...

[0013] The region corresponding to the target coordinates is cropped to obtain the target region;

[0014] The target category is used as the label of the target region, and the image classification network in the teacher model with randomized weight coefficients is trained to obtain the teacher model with updated weight coefficients.

[0015] In one embodiment of the present invention, after obtaining the teacher model with updated weight coefficients, the method further includes:

[0016] The training loss value of the object detection network in the teacher model is calculated based on the first loss function of the object detection network and the weight coefficients are updated.

[0017] The training loss value of the image classification network in the teacher model, which updates the weight coefficients, is calculated based on the image classification network loss function.

[0018] In one embodiment of the present invention, the calculation formula for the first loss function of the target detection network is as follows:

[0019]

[0020] Among them, Loss detectionThe value represents the training loss of the object detection network; i represents the i-th bounding box; l i This represents the target coordinates of the predicted i-th bounding box; The coordinates represent the actual target coordinates; x, y, w, and h represent the x and y coordinates of the top-left vertex of the target region, the width of the region, and the height of the region, respectively. The actual label representing the category; y i Indicates the predicted category; smooth L1 Used to measure l i and The difference; softmax(y) represents the normalization operation when predicting the category;

[0021] The formula for calculating the loss function of the image classification network is as follows:

[0022]

[0023] Among them, Loss classification This represents the training loss value of the image classification network; The actual label representing the category; y i The value represents the predicted category; softmax(y) represents the normalization operation when predicting the category.

[0024] In one embodiment of the present invention, the teacher model updated with the weight coefficients is used to predict target objects in the unlabeled dataset to obtain pseudo-target coordinates and pseudo-target categories, including:

[0025] The target detection network in the teacher model updated with the weight coefficients is used to predict the target objects in the unlabeled dataset to obtain the coordinates of the pseudo-targets;

[0026] The region corresponding to the coordinates of the pseudo-target is cropped to obtain the pseudo-target region;

[0027] The pseudo-target region is predicted by the image classification network in the teacher model updated with the weight coefficients, and the pseudo-target category is obtained.

[0028] In one embodiment of the present invention, after obtaining the student model with updated weight coefficients, the method further includes:

[0029] The training loss value of the object detection network in the student model with updated weight coefficients is calculated based on the second loss function of the object detection network. The training loss value of the object detection network in the student model with updated weight coefficients includes the detection loss value.

[0030] The training loss value of the image classification network in the student model with updated weight coefficients is calculated based on the image classification network loss function.

[0031] In one embodiment of the present invention, the weight coefficients of the teacher model updated with the updated weight coefficients are updated based on the weight coefficients of the student model, until the weight coefficients between the two meet a preset condition, and the student model with the updated weight coefficients is used as a semi-supervised object detection model, including:

[0032] When the training loss value of the image classification network in the student model with updated weight coefficients is less than a first preset threshold, the training loss value of the target detection network in the student model with updated weight coefficients is calculated based on the first loss function of the target detection network, so that the training loss value of the target detection network in the student model with updated weight coefficients includes the detection loss value and the category loss value.

[0033] The weight coefficients of the teacher model are updated based on the weight coefficients of the student model after the weight coefficients of the object detection network loss function are updated.

[0034] When the difference between the weight coefficient of the updated student model and the weight coefficient of the updated teacher model is less than a second preset threshold, the currently updated student model is used as a semi-supervised target detection model.

[0035] In one embodiment of the present invention, the object detection network loss function in the student model that previously updated the weight coefficients is modified to the object detection network second loss function, and the calculation formula is as follows:

[0036]

[0037] Among them, Loss detection The value represents the training loss of the object detection network; i represents the i-th bounding box; l i This represents the target coordinates of the predicted i-th bounding box; Indicates the corresponding true target coordinates; x, y, w, h represent the x and y coordinates of the top-left vertex of the target region, the width and height of the region, respectively; smooth L1 Used to measure l i and The difference;

[0038] In the modified student model with updated weight coefficients, the loss function for the object detection network is the first loss function for the object detection network, calculated as follows:

[0039]

[0040] Among them, Loss detection The value represents the training loss of the object detection network; i represents the i-th bounding box; l i This represents the target coordinates of the predicted i-th bounding box; The coordinates represent the actual target coordinates; x, y, w, and h represent the x and y coordinates of the top-left vertex of the target region, the width of the region, and the height of the region, respectively. The actual label representing the category; y i Indicates the predicted category; smooth L1 Used to measure l i and The difference; softmax(y) represents the normalization operation when predicting the category.

[0041] In one embodiment of the present invention, after using the student model with the currently updated weight coefficients as a semi-supervised object detection model, the method further includes:

[0042] The image to be tested is input into the semi-supervised target detection model to obtain the detection result of the image to be tested.

[0043] The present invention also provides a semi-supervised object detection model training device, characterized in that the device comprises:

[0044] The dataset acquisition module is used to acquire labeled datasets and unlabeled datasets, wherein the image samples in the labeled datasets carry the target coordinates and target category of the target object;

[0045] The teacher model training module is used to construct a teacher model and a student model with randomized weight coefficients, which contain the same object detection network and image classification network. The teacher model with randomized weight coefficients is trained using the labeled dataset to obtain a teacher model with updated weight coefficients.

[0046] The prediction module is used to predict target objects in the unlabeled dataset using the teacher model updated with the weight coefficients, and to obtain pseudo-target coordinates and pseudo-target categories.

[0047] The student model training module is used to jointly train the student model with randomized weight coefficients based on the target coordinates, the target category, the pseudo-target coordinates, and the pseudo-target category, so as to obtain a student model with updated weight coefficients.

[0048] The weight coefficient update module is used to update the weight coefficient of the teacher model based on the weight coefficient of the student model updated by the weight coefficient, until the weight coefficient between the two meets the preset conditions, and the student model updated by the weight coefficient is used as a semi-supervised object detection model.

[0049] The present invention also provides an electronic device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the above-described method.

[0050] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method.

[0051] As described above, the semi-supervised target detection model training method, apparatus, device, and medium provided in this embodiment of the invention have the following beneficial effects:

[0052] First, labeled and unlabeled datasets are obtained. The labeled dataset contains image samples carrying the target coordinates and category of the target object. Then, a teacher model with randomized weight coefficients is trained using the labeled dataset to obtain a teacher model with updated weight coefficients. Next, the teacher model with updated weight coefficients is used to predict target objects in the unlabeled dataset, obtaining pseudo-target coordinates and pseudo-target categories. Then, a student model with randomized weight coefficients is jointly trained based on the target coordinates, target category, pseudo-target coordinates, and pseudo-target categories to obtain a student model with updated weight coefficients. Finally, the weight coefficients of the teacher model with updated weight coefficients are updated based on the weight coefficients of the student model until the weight coefficients between the two meet a preset condition. The student model with updated weight coefficients is then used as the semi-supervised object detection model. By splitting the detection task into two sub-tasks—localization and classification—and training the model accordingly, an accurate and highly generalizable semi-supervised object detection model can be obtained.

[0053] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0055] Figure 1 This is a schematic diagram illustrating the implementation environment of a semi-supervised object detection model training device, as shown in an exemplary embodiment of this application.

[0056] Figure 2 This is a flowchart illustrating a semi-supervised object detection model training method in an exemplary embodiment of this application;

[0057] Figure 3 This is a flowchart illustrating a specific semi-supervised object detection model training method as shown in an exemplary embodiment of this application;

[0058] Figure 4This is a block diagram illustrating a semi-supervised object detection model training apparatus as shown in an exemplary embodiment of this application;

[0059] Figure 5 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0060] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and sub-samples in the embodiments can be combined with each other.

[0061] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0062] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0063] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0064] Unless otherwise stated, the term "multiple" means two or more.

[0065] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0066] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0067] Object detection has a wide range of applications, such as autonomous driving, industrial production, video surveillance, medical image processing, and satellite image processing. Most object detection methods rely on semi-supervised object detection models to predict the target object. However, when using pseudo-label learning in a semi-supervised model, the target detection network directly predicts both the object's coordinates and category simultaneously. This results in a lower ability to classify unlabeled data, impacting the accuracy of the object detection results.

[0068] To resolve the above issues, please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram illustrating the implementation environment of a semi-supervised object detection model training device, as shown in an exemplary embodiment of this application. Typically, in the field of autonomous driving, object detection, as the core foundation of autonomous driving perception systems, is a prerequisite for functions such as path planning, motion prediction, and collision avoidance. To ensure the reliability and safety of autonomous driving, a highly accurate and generalizable semi-supervised object detection model is needed to predict objects in the unlabeled image data acquired in real time during autonomous driving. Figure 1 As shown in the diagram, the real-time environment includes a cloud platform 101 and a vehicle platform 102, used for training a semi-supervised object detection model. The vehicle platform 102 uploads the collected images to the cloud platform 101, which processes the images to obtain an image dataset. The cloud platform 101 is primarily responsible for training the model based on both labeled and unlabeled image datasets to obtain a semi-supervised object detection model, enabling accurate prediction of the target object in the images collected by the vehicle platform 101 during autonomous driving.

[0069] Please see Figure 2 , Figure 2 This is a flowchart illustrating a semi-supervised object detection model training method in an exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown is specifically executed by a client and / or server within that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and executed by devices in other implementation environments; this embodiment does not limit the implementation environment to which the method is applicable. To address these issues, embodiments of this application propose a semi-supervised object detection model training method, a semi-supervised object detection model training apparatus, a device, and a computer-readable storage medium, which will be described in detail below.

[0070] like Figure 2 As shown, in an exemplary embodiment, the semi-supervised object detection model training method includes at least steps S201 to S205, which are described in detail below:

[0071] Step S201: Obtain the labeled dataset and the unlabeled dataset. The image samples in the labeled dataset carry the target coordinates and target category of the target object.

[0072] Labeled data enables the identification of a given unlabeled image. Unlabeled data doesn't define the meaning of an image; it simply contains the data. For example, labeled data might be labeled as "the photo contains a horse," while unlabeled data might be labeled as "whether the photo contains a horse." Both labeled and unlabeled datasets can be used as base data to train a semi-supervised object detection model. Then, new unlabeled data can be presented to the model to predict its label. It should be understood that image samples in the labeled dataset carry the coordinates and category of the target object. When training a semi-supervised object detection model using the labeled dataset, it can predict the coordinates and category of the target object in image samples from the unlabeled dataset.

[0073] Step S202: Construct a teacher model and a student model with randomized weight coefficients, both containing the same object detection network and image classification network. Train the teacher model with randomized weight coefficients using a labeled dataset to obtain a teacher model with updated weight coefficients.

[0074] To address the issue of low accuracy in detecting both target coordinates and categories of objects simultaneously through an object detection network, a teacher model with randomized weight coefficients and a student model with randomized weight coefficients, both containing the same object detection network and image classification network, were constructed. Then, the object detection network and image classification network in the teacher model with randomized weight coefficients were trained using the target coordinates and categories of objects from the labeled dataset. This resulted in a teacher model with updated weight coefficients, enabling the object detection network in the updated teacher model to more accurately predict the coordinates of objects in unlabeled images, and the image classification network to more accurately predict the categories of objects in unlabeled images.

[0075] It should be understood that object detection networks can independently predict the coordinates and category of the target object; that is, they possess both detection and classification capabilities. However, predicting both coordinates and category simultaneously can affect the accuracy of category prediction. Therefore, image classification networks were constructed for both the teacher model and the student model with randomized weight coefficients to predict the target object's category separately. These image classification networks replace the classification function in the object detection model, improving the accuracy of the model's category prediction. Furthermore, in the early stages of training when prediction performance on unlabeled data is poor, the learning difficulty for the student model with randomized weight coefficients can be reduced.

[0076] In one embodiment, a teacher model with randomized weight coefficients is trained using a labeled dataset to obtain a teacher model with updated weight coefficients, including:

[0077] The target detection network in the teacher model with randomized weight coefficients is trained using target coordinates and target category; and...

[0078] The target region is obtained by cropping the area corresponding to the target coordinates.

[0079] By using the target category as the label for the target region, the image classification network in the teacher model with randomized weight coefficients is trained to obtain a teacher model with updated weight coefficients.

[0080] In the process of training a teacher model with randomized weight coefficients using a labeled dataset, the target detection network in the teacher model is first trained using the target coordinates and target categories of the target objects in the labeled dataset. Then, the regions corresponding to the target coordinates are cropped to obtain the target regions. Next, the target categories are used as labels for the target regions to train the image classification network in the teacher model with randomized weight coefficients, resulting in a teacher model with updated weight coefficients. In the obtained teacher model with updated weight coefficients, the target detection network can more accurately predict the coordinates of target objects in unlabeled data images, and the image classification network can more accurately predict the categories of target objects in unlabeled data images.

[0081] In one embodiment, after obtaining the teacher model with updated weight coefficients, the method further includes:

[0082] The training loss value of the object detection network in the teacher model is calculated based on the first loss function of the object detection network to update the weight coefficients;

[0083] The training loss value of the image classification network in the teacher model is calculated based on the loss function of the image classification network to update the weight coefficients.

[0084] It should be understood that after training the object detection network in the teacher model with randomized weight coefficients to obtain the teacher model with updated weight coefficients, there is a deviation between the object detection network's prediction of the data and the true label of the data in the teacher model with updated weight coefficients. This represents the accuracy of the model's prediction for a single sample. If the deviation between the object detection network's prediction of the data and the true label of the data in the teacher model with updated weight coefficients is large, the loss value will also be large; if the deviation is small or almost the same, the loss value will be small. Similarly, after training the image classification network in the teacher model with randomized weight coefficients to obtain the teacher model with updated weight coefficients, there is also a deviation between the image classification network's prediction of the data and the true label of the data in the teacher model with updated weight coefficients. Therefore, it is necessary to calculate the training loss value of the object detection network in the teacher model with updated weight coefficients and the training loss value of the image classification network in the teacher model with updated weight coefficients. Then, based on the training loss value, the difference between the forward calculation result and the true value of the teacher model with updated weight coefficients in each iteration can be determined, thereby optimizing the training direction and reducing the training loss value.

[0085] In one embodiment, the formula for calculating the first loss function of the object detection network is:

[0086]

[0087] Among them, Loss detection The value represents the training loss of the object detection network; i represents the i-th bounding box; l i This represents the target coordinates of the predicted i-th bounding box; The coordinates represent the actual target coordinates; x, y, w, and h represent the x and y coordinates of the top-left vertex of the target region, the width of the region, and the height of the region, respectively. The actual label representing the category; y i Indicates the predicted category; smooth L1 Used to measure l i and The difference; softmax(y) represents the normalization operation when predicting the category;

[0088] The formula for calculating the loss function of an image classification network is:

[0089]

[0090] Among them, Loss classification This represents the training loss value of the image classification network; The actual label representing the category; y i The value represents the predicted category; softmax(y) represents the normalization operation when predicting the category.

[0091] It is important to note that because object detection networks have both detection and classification capabilities, the training loss of an object detection network for a teacher model that updates weight coefficients includes both detection loss and classification loss.

[0092] Step S203: Use the teacher model updated with weight coefficients to predict the target objects in the unlabeled dataset and obtain the pseudo-target coordinates and pseudo-target categories.

[0093] The teacher model for updating weight coefficients is obtained by training a teacher model with randomized weight coefficients using the target coordinates and categories of target objects in the labeled dataset. It can be considered that the teacher model for updating weight coefficients has the ability to predict unlabeled datasets and can accurately predict the pseudo-target coordinates and pseudo-target categories of target objects in the unlabeled dataset. Therefore, the object detection network and image classification network in the teacher model for updating weight coefficients are used to predict the unlabeled dataset to obtain the pseudo-target coordinates and pseudo-target categories of target objects in the unlabeled dataset. It should be understood that the aforementioned pseudo-target coordinates and pseudo-target categories are pseudo-labels, which are the results of predictions made on the unlabeled dataset using a model trained on the labeled dataset.

[0094] In one embodiment, a teacher model updated with weight coefficients is used to predict target objects in an unlabeled dataset to obtain pseudo-target coordinates and pseudo-target categories, including:

[0095] The target detection network in the teacher model, which uses weighted coefficient updates, predicts target objects in the unlabeled dataset to obtain pseudo-target coordinates.

[0096] The pseudo-target region is obtained by cropping the region corresponding to the coordinates of the pseudo-target.

[0097] The image classification network in the teacher model, which updates the weight coefficients, predicts the pseudo-target region and obtains the pseudo-target category.

[0098] To predict target objects in an unlabeled dataset, we can obtain the pseudo-target coordinates and pseudo-target categories. First, the target detection network in the teacher model with updated weight coefficients predicts the target objects in the unlabeled dataset to obtain the pseudo-target coordinates. Then, the regions corresponding to the pseudo-target coordinates are cropped to obtain the pseudo-target regions. After obtaining the pseudo-target regions, the image classification network in the teacher model with updated weight coefficients predicts the pseudo-target regions to obtain the pseudo-target categories.

[0099] Step S204: Train the student model with randomized weight coefficients jointly based on the target coordinates, target category, pseudo-target coordinates, and pseudo-target category to obtain the student model with updated weight coefficients.

[0100] By training the object detection network and image classification network in the student model with randomized weight coefficients using the target coordinates and target category of the target object in the labeled dataset, and the pseudo-target coordinates and pseudo-target category of the target object in the unlabeled dataset, a student model with updated weight coefficients is obtained. This allows the object detection network and image classification network in the student model with updated weight coefficients to accurately predict the coordinates and category of the target object.

[0101] In one embodiment, after obtaining the updated student model with weight coefficients, the method further includes:

[0102] The training loss value of the object detection network in the student model with updated weight coefficients is calculated based on the second loss function of the object detection network. The training loss value of the object detection network in the student model with updated weight coefficients includes the detection loss value.

[0103] The training loss value of the image classification network in the student model is calculated based on the loss function of the image classification network to update the weight coefficients.

[0104] It should be understood that after training the object detection network in the student model with randomized weight coefficients to obtain a student model with updated weight coefficients, there is a discrepancy between the object detection network's predictions and the true labels of the data; similarly, after training the image classification network in the student model with randomized weight coefficients to obtain a student model with updated weight coefficients, there is also a discrepancy between the image classification network's predictions and the true labels of the data. Therefore, it is necessary to calculate the training loss values ​​of the object detection network and the image classification network in the student model with updated weight coefficients, and then modify the weight coefficients of the student model with updated weight coefficients based on the training loss values ​​to optimize the student model. During the training process of the student model with updated weight coefficients, the training loss values ​​of the object detection network and the image classification network can be gradually reduced.

[0105] Step S205: Based on the updated weight coefficients of the student model, update the weight coefficients of the updated teacher model until the weight coefficients between the two meet the preset conditions, and use the student model with the updated weight coefficients as the semi-supervised object detection model.

[0106] After adjusting the training loss values ​​of the target detection network and the image classification network based on the updated weight coefficients of the student model, the weight coefficients of the updated weight coefficients are used to update the weight coefficients of the updated weight coefficients of the teacher model. The teacher model and the student model are then iteratively trained. When the weight coefficients of the updated weight coefficients of the student model and the teacher model meet preset conditions, it can be considered that the updated weight coefficients of the student model can accurately predict target objects in both new labeled and unlabeled datasets. In this case, the current updated weight coefficients of the student model are used as a semi-supervised target detection model for target detection.

[0107] The formula for calculating the weight coefficients of the teacher model based on the weight coefficient update of the student model is as follows:

[0108] θ teacher =αθ teacher +(1-α)θ student

[0109] In the formula, θ teach θ represents the weight coefficients of the teacher model used for weight coefficient updates. student This represents the weight coefficients of the student model whose weight coefficients are updated; α is a constant value. A larger α indicates a smaller update magnitude for the teacher model's weight coefficients, signifying a more stable teacher model with slower convergence; conversely, a smaller α indicates a larger update magnitude for the teacher model's weight coefficients, signifying a less stable teacher model with faster convergence.

[0110] In one embodiment, the weight coefficients of the student model are updated based on the updated weight coefficients, and the weight coefficients of the teacher model are updated based on the updated weight coefficients, until the weight coefficients between the two meet a preset condition. The student model with the currently updated weight coefficients is then used as a semi-supervised object detection model, including:

[0111] When the training loss value of the image classification network in the student model with updated weight coefficients is less than the first preset threshold, the training loss value of the object detection network in the student model with updated weight coefficients is calculated based on the first loss function of the object detection network, so that the training loss value of the object detection network in the student model with updated weight coefficients includes the detection loss value and the category loss value.

[0112] The weight coefficients of the student model are updated based on the weight coefficients after the loss function of the object detection network is modified, and the weight coefficients of the teacher model are updated based on the weight coefficients of the teacher model.

[0113] When the difference between the weight coefficient of the updated student model and the weight coefficient of the updated teacher model is less than the second preset threshold, the student model with the updated weight coefficient is used as a semi-supervised object detection model.

[0114] When training the teacher model with updated weight coefficients using target coordinates, target category, pseudo-target coordinates, and pseudo-target category, the accuracy of the pseudo-target category is initially low. Therefore, the training loss value of the object detection network in the student model with updated weight coefficients only includes the detection loss value, excluding the category loss value. However, during iterative training between the teacher model and the student model with updated weight coefficients, the prediction accuracy for the category improves. When the training loss value of the image classification network in the student model with updated weight coefficients is less than a first preset threshold, the training loss value of the object detection network in the student model with updated weight coefficients is calculated based on the first loss function of the object detection network. That is, the training loss value of the object detection network in the student model with updated weight coefficients includes both the detection loss value and the category loss value. Then, the weight coefficients of the teacher model with updated weight coefficients are updated based on the weight coefficients of the student model with updated weight coefficients after modifying the object detection network loss function, until the difference between the weight coefficients of the student model with updated weight coefficients and the weight coefficients of the teacher model with updated weight coefficients is less than a second preset threshold, i.e., θ. teacher -θ student When T < T, it can be assumed that the student model with updated weight coefficients can make relatively accurate predictions for target objects in both the new labeled dataset and the unlabeled dataset. Therefore, the student model with updated weight coefficients is regarded as a semi-supervised object detection model.

[0115] In one embodiment, the object detection network loss function in the student model that previously updated the weight coefficients is modified to the object detection network's second loss function, calculated as follows:

[0116]

[0117] Among them, Loss detection The value represents the training loss of the object detection network; i represents the i-th bounding box; l i This represents the target coordinates of the predicted i-th bounding box; Indicates the corresponding true target coordinates; x, y, w, h represent the x and y coordinates of the top-left vertex of the target region, the width and height of the region, respectively; smooth L1 Used to measure l i and The difference;

[0118] In the modified student model with updated weight coefficients, the loss function for the object detection network is the first loss function for the object detection network, calculated as follows:

[0119]

[0120] Among them, Loss detection The value represents the training loss of the object detection network; i represents the i-th bounding box; l i This represents the target coordinates of the predicted i-th bounding box; The coordinates represent the actual target coordinates; x, y, w, and h represent the x and y coordinates of the top-left vertex of the target region, the width of the region, and the height of the region, respectively. The actual label representing the category; y i Indicates the predicted category; smooth L1 Used to measure l i and The difference; softmax(y) represents the normalization operation when predicting the category.

[0121] In one embodiment, after using the student model with the currently updated weight coefficients as a semi-supervised object detection model, the method further includes:

[0122] The image to be tested is input into a semi-supervised object detection model to obtain the detection result of the image to be tested.

[0123] After obtaining a trained semi-supervised detection model, the target object in a new test image can be predicted based on the semi-supervised object detection model. That is, the test image is directly input into the semi-supervised object detection model, and the detection result is output by the semi-supervised object detection model.

[0124] Please see Figure 3 , Figure 3 This is a flowchart illustrating a specific semi-supervised object detection model training method, as shown in an exemplary embodiment of this application. Figure 3 As shown,

[0125] First, the target detection network of the teacher model with randomized weight coefficients is trained using a labeled dataset containing the target coordinates and target category of the target object. Then, the target region is obtained by cropping the region corresponding to the target coordinates and the target category is used as the label of the target region to train the image classification network of the teacher model with randomized weight coefficients.

[0126] Then, the teacher model with updated weight coefficients obtained after training is used to predict the unlabeled dataset. Specifically, the target detection network in the teacher model with updated weight coefficients obtained after training is used to obtain the coordinates of the target objects in the unlabeled dataset, and the regions corresponding to the pseudo-target coordinates are cropped to obtain pseudo-target regions. Then, the image classification network in the teacher model with updated weight coefficients obtained after training is used to predict the pseudo-target regions to obtain the pseudo-target categories.

[0127] Next, the target detection network in the student model with randomized weight coefficients is trained by using the target coordinates of the target objects in the labeled dataset and the pseudo-target coordinates of the target objects in the unlabeled dataset. The image classification network in the student model with randomized weight coefficients is trained by using the target categories of the target objects in the labeled dataset and the pseudo-target categories of the target objects in the unlabeled dataset. This results in a student model with updated weight coefficients after training.

[0128] Furthermore, the weight coefficients of the student model updated with the trained weight coefficients are used to update the weight coefficients of the teacher model updated with the trained weight coefficients. When the training loss value of the image classification network in the student model updated with the trained weight coefficients is less than the first preset threshold, the loss function for calculating the training loss value of the object detection network in the student model updated with the trained weight coefficients is modified, and iterative training of the teacher model and the student model updated with the trained weight coefficients continues.

[0129] Finally, it is determined whether the difference between the updated weight coefficients of the teacher model and the updated weight coefficients of the student model is less than a second preset threshold T. If the difference is not less than the second preset threshold T, iterative training of both models continues, i.e., the weight coefficients of the updated teacher model are updated using the weight coefficients of the updated student model. If the difference is less than the second preset threshold T, training ends, and the currently updated student model is used as the semi-supervised object detection model.

[0130] Please see Figure 4 , Figure 4 This is a block diagram illustrating a semi-supervised object detection model training apparatus, as shown in an exemplary embodiment of this application. Figure 4 As shown, this embodiment provides a semi-supervised object detection model training device 400, which includes:

[0131] The dataset acquisition module 401 is used to acquire labeled datasets and unlabeled datasets. The image samples in the labeled datasets carry the target coordinates and target category of the target object.

[0132] The teacher model training module 402 is used to construct a teacher model with randomized weight coefficients and a student model with randomized weight coefficients, which contain the same object detection network and image classification network. The teacher model with randomized weight coefficients is trained using a labeled dataset to obtain a teacher model with updated weight coefficients.

[0133] Prediction module 403 is used to predict target objects in the unlabeled dataset using a teacher model updated with weight coefficients, and to obtain pseudo-target coordinates and pseudo-target categories.

[0134] The student model training module 404 is used to jointly train a student model with randomized weight coefficients based on the target coordinates, target category, pseudo-target coordinates, and pseudo-target category, and obtain a student model with updated weight coefficients.

[0135] The weight coefficient update module 405 is used to update the weight coefficients of the student model based on the updated weight coefficients, and update the weight coefficients of the teacher model based on the updated weight coefficients, until the weight coefficients between the two meet the preset conditions, and then use the student model with the updated weight coefficients as a semi-supervised object detection model.

[0136] Please see Figure 5 , Figure 5 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 5 The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0137] like Figure 5 As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage portion 508 into Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0138] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0139] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0140] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0141] This disclosure also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements any of the methods in this embodiment.

[0142] The computer-readable storage medium in the embodiments of this disclosure will be understood by those skilled in the art: all or part of the steps of the above method embodiments can be implemented by hardware related to computer programs. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0143] The electronic device disclosed in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic device performs the various steps of the above method.

[0144] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0145] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), graphics processing units (GPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0146] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and subsamples of some embodiments may be included in or replace parts and subsamples of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used herein means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated subsamples, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other subsamples, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes the element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0147] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0148] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some sub-samples may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A semi-supervised object detection model training method, characterized in that, The method includes: Obtain labeled and unlabeled datasets, wherein the image samples in the labeled dataset carry the target coordinates and target category of the target object; Construct a teacher model and a student model with randomized weight coefficients, both containing the same object detection network and image classification network. Train the teacher model with randomized weight coefficients using the labeled dataset to obtain a teacher model with updated weight coefficients. The teacher model updated with the weight coefficients is used to predict target objects in the unlabeled dataset to obtain pseudo-target coordinates and pseudo-target categories; The student model with randomized weight coefficients is jointly trained based on the target coordinates, the target category, the pseudo-target coordinates, and the pseudo-target category to obtain a student model with updated weight coefficients. When the training loss value of the image classification network in the student model with updated weight coefficients is less than a first preset threshold, the training loss value of the object detection network in the student model with updated weight coefficients is calculated based on the first loss function of the object detection network, so that the training loss value of the object detection network in the student model with updated weight coefficients includes the detection loss value and the category loss value; based on the weight coefficients of the student model with updated weight coefficients after modifying the loss function of the object detection network, the weight coefficients of the teacher model with updated weight coefficients are updated; until the weight coefficients between the two meet the preset conditions, the current student model with updated weight coefficients is used as a semi-supervised object detection model, and the semi-supervised object detection model is used to detect the image to be tested and output the detection result of the image to be tested.

2. The semi-supervised target detection model training method according to claim 1, characterized in that, The teacher model with randomized weight coefficients is trained using the labeled dataset to obtain a teacher model with updated weight coefficients, including: The target detection network in the teacher model with randomized weight coefficients is trained using the target coordinates and the target category; and... The region corresponding to the target coordinates is cropped to obtain the target region; The target category is used as the label of the target region, and the image classification network in the teacher model with randomized weight coefficients is trained to obtain the teacher model with updated weight coefficients.

3. The semi-supervised target detection model training method according to claim 1 or 2, characterized in that, After obtaining the updated teacher model with the weight coefficients, the method further includes: The training loss value of the object detection network in the teacher model is calculated based on the first loss function of the object detection network and the weight coefficients are updated. The training loss value of the image classification network in the teacher model, which updates the weight coefficients, is calculated based on the image classification network loss function.

4. The semi-supervised target detection model training method according to claim 3, characterized in that, The formula for calculating the first loss function of the target detection network is as follows: in, This represents the training loss value of the object detection network; i This represents the i-th frame; Indicates the predicted first The target coordinates of each box; This represents the corresponding actual target coordinates; x , y , w , h These represent the x and y coordinates of the top-left vertex of the target region, as well as the width and height of the region, respectively. The actual label representing the category; Indicates the category of the prediction; smooth L1 Used for measurement and The difference; softmax ( y () represents the normalization operation when predicting categories; The formula for calculating the loss function of the image classification network is as follows: in, This represents the training loss value of the image classification network; The actual label representing the category; Indicates the category of the prediction; softmax ( y ) represents the normalization operation when predicting categories.

5. The semi-supervised target detection model training method according to claim 1, characterized in that, The teacher model, updated using the aforementioned weight coefficients, predicts target objects in the unlabeled dataset to obtain pseudo-target coordinates and pseudo-target categories, including: The target detection network in the teacher model updated with the weight coefficients is used to predict the target objects in the unlabeled dataset to obtain the coordinates of the pseudo-targets; The region corresponding to the coordinates of the pseudo-target is cropped to obtain the pseudo-target region; The pseudo-target region is predicted by the image classification network in the teacher model updated with the weight coefficients, and the pseudo-target category is obtained.

6. The semi-supervised target detection model training method according to claim 1, characterized in that, After obtaining the updated student model with weight coefficients, the method further includes: The training loss value of the object detection network in the student model with updated weight coefficients is calculated based on the second loss function of the object detection network. The training loss value of the object detection network in the student model with updated weight coefficients includes the detection loss value. The training loss value of the image classification network in the student model with updated weight coefficients is calculated based on the image classification network loss function.

7. The semi-supervised target detection model training method according to claim 1, characterized in that, Until the weight coefficients between the two meet preset conditions, the student model updated with the current weight coefficients is used as a semi-supervised object detection model, including: When the difference between the weight coefficient of the updated student model and the weight coefficient of the updated teacher model is less than a second preset threshold, the currently updated student model is used as a semi-supervised target detection model.

8. The semi-supervised target detection model training method according to claim 7, characterized in that, The object detection network loss function in the previously modified student model for updating weight coefficients has been changed to the object detection network's second loss function, calculated as follows: in, This represents the training loss value of the object detection network; i This represents the i-th frame; Indicates the predicted first The target coordinates of each box; This represents the corresponding actual target coordinates; x , y , w , h These represent the x and y coordinates of the top-left vertex of the target region, as well as the width and height of the region, respectively. smooth L1 Used for measurement and The difference; In the modified student model with updated weight coefficients, the loss function for the object detection network is the first loss function for the object detection network, and the calculation formula is: in, This represents the training loss value of the object detection network; i This represents the i-th frame; Indicates the predicted first The target coordinates of each box; This represents the corresponding actual target coordinates; x , y , w , h These represent the x and y coordinates of the top-left vertex of the target region, as well as the width and height of the region, respectively. The actual label representing the category; Indicates the category of the prediction; smooth L1 Used for measurement and The difference; softmax ( y ) represents the normalization operation when predicting categories.

9. A semi-supervised target detection model training device, characterized in that, The device includes: The dataset acquisition module is used to acquire labeled datasets and unlabeled datasets, wherein the image samples in the labeled datasets carry the target coordinates and target category of the target object; The teacher model training module is used to construct a teacher model and a student model with randomized weight coefficients, which contain the same object detection network and image classification network. The teacher model with randomized weight coefficients is trained using the labeled dataset to obtain a teacher model with updated weight coefficients. The prediction module is used to predict target objects in the unlabeled dataset using the teacher model updated with the weight coefficients, and to obtain pseudo-target coordinates and pseudo-target categories. The student model training module is used to jointly train the student model with randomized weight coefficients based on the target coordinates, the target category, the pseudo-target coordinates, and the pseudo-target category, so as to obtain a student model with updated weight coefficients. The weight coefficient update module is used to calculate the training loss value of the object detection network in the student model with updated weight coefficients based on the first loss function of the object detection network when the training loss value of the image classification network in the student model with updated weight coefficients is less than a first preset threshold, so that the training loss value of the object detection network in the student model with updated weight coefficients includes the detection loss value and the category loss value; based on the weight coefficients of the student model with updated weight coefficients after modifying the loss function of the object detection network, update the weight coefficients of the teacher model with updated weight coefficients; until the weight coefficients between the two meet the preset conditions, the current student model with updated weight coefficients is used as a semi-supervised object detection model, and the semi-supervised object detection model is used to detect the image to be tested and output the detection result of the image to be tested.

10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the method of any one of claims 1 to 8.

Images