Training method and device of target detection model, computer device and storage medium

Abstract

Embodiments of the present application disclose a target detection model training method and device, computer equipment and a storage medium. Embodiments of the present application input a first image sample into a pre-trained detection model, output a corresponding first target detection result, determine a first target loss function according to the first target detection result and label data carried by the first image sample, input a second image sample into a teacher detection model and a student detection model respectively, output a corresponding second target detection result and a third target detection result to determine a second target loss function, update first network parameters of the student detection model based on the first target loss function and the second target loss function, update second network parameters of the teacher detection model according to the updated first network parameters, continuously update the second network parameters until an iteration condition is met to stop, and determine the trained teacher detection model as the target detection model. The efficiency of data processing is greatly improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, specifically to a training method, apparatus, computer device, and storage medium for an object detection model. Background Technology

[0002] In recent years, deep learning network models have become the most practical models in application fields such as computer vision. The success of deep learning is mainly attributed to its scalability, that is, the existence of a large amount of labeled data is required for deep learning network models to exhibit robust performance.

[0003] In the process of researching and practicing the prior art, the inventors of this application discovered that in the prior art, a large amount of high-quality label data means that a large amount of manual annotation is required, and the cost of manual annotation is extremely high, especially the cost of data annotation by experts. Moreover, the speed of manual annotation is often slow and the efficiency of data processing is poor. Summary of the Invention

[0004] This application provides a training method, apparatus, computer device, and storage medium for an object detection model, which can improve the training efficiency of the object detection model.

[0005] To address the aforementioned technical problems, this application provides the following technical solutions:

[0006] A method for training an object detection model, comprising:

[0007] Acquire a first image sample and a second image sample, wherein the first image sample carries label data and the second image sample does not carry label data;

[0008] The first image sample is input into the pre-trained detection model, and the corresponding first target detection result is output.

[0009] The first target loss function is determined based on the first target detection result and the label data carried by the first image sample;

[0010] The second image sample is input into the teacher detection model and the student detection model respectively, and the corresponding second target detection result and third target detection result are output.

[0011] The second target loss function is determined based on the difference between the second target detection result and the third target detection result;

[0012] The first network parameters of the student detection model are updated based on the first objective loss function and the second objective loss function;

[0013] The second network parameters of the teacher detection model are updated according to the updated first network parameters, and the process iteratively returns to the process of inputting the first image sample into the pre-trained detection model and outputting the corresponding first target detection result, so as to continuously update the second network parameters until the iteration condition is met and the trained teacher detection model is determined as the target detection model.

[0014] A data processing method, comprising:

[0015] The image to be labeled is input into the target detection model trained by the above target detection model training method, and the corresponding fourth target detection result is output.

[0016] Based on the fourth target detection result, the target label information corresponding to the image to be labeled is generated.

[0017] A training device for an object detection model, comprising:

[0018] The acquisition unit is used to acquire a first image sample and a second image sample, wherein the first image sample carries label data and the second image sample does not carry label data;

[0019] The first output unit is used to input the first image sample into the pre-trained detection model and output the corresponding first target detection result;

[0020] The first determining unit is configured to determine a first target loss function based on the first target detection result and the label data carried by the first image sample;

[0021] The second output unit is used to input the second image sample into the teacher detection model and the student detection model respectively, and output the corresponding second target detection result and the third target detection result.

[0022] The second determining unit is used to determine the second target loss function based on the difference between the second target detection result and the third target detection result;

[0023] The first update unit is used to update the first network parameters of the student detection model based on the first target loss function and the second target loss function;

[0024] The iterative update unit is used to update the second network parameters of the teacher detection model according to the updated first network parameters, and iteratively return to execute the input of the first image sample into the pre-trained detection model and output the corresponding first target detection result, so as to continuously update the second network parameters until the iteration condition is met and the trained teacher detection model is determined as the target detection model.

[0025] In some embodiments, the apparatus further includes:

[0026] The third output unit is used to input the first image sample into the detection model and output the corresponding first detection result.

[0027] The third determining unit is used to determine a third target loss function based on the difference between the first detection result and the label data;

[0028] The second update unit is used to update the third network parameters of the detection model based on the third objective loss function, and iteratively return to execute the input of the first image sample into the detection model and the output of the corresponding first detection result, so as to continuously update the third network parameters until the third objective loss function converges and a pre-trained detection model is obtained.

[0029] The transmission unit is used to transmit the first target network parameters of the pre-trained detection model to the teacher detection model and the student detection model.

[0030] In some embodiments, the third determining unit is configured to:

[0031] The first loss function is determined based on the difference between the classification information of the first region in the first detection result and the classification information of the label region in the label data;

[0032] The second loss function is determined based on the difference between the location information of the first region in the first detection result and the location information of the label region in the label data;

[0033] The third loss function is determined based on the difference between the classification information of the first region of interest in the first detection result and the classification information of the label region of interest in the label data;

[0034] The fourth loss function is determined based on the difference between the location information of the first region of interest in the first detection result and the location information of the region of interest of the label in the label data;

[0035] A third objective loss function is constructed based on the first, second, third, and fourth loss functions.

[0036] In some embodiments, the first determining unit is configured to:

[0037] The fifth loss function is determined based on the difference between the classification information of the second region in the first target detection result and the classification information of the label region in the label data;

[0038] The sixth loss function is determined based on the difference between the location information of the second region in the first target detection result and the location information of the label region in the label data;

[0039] The seventh loss function is determined based on the difference between the classification information of the second region of interest in the first target detection result and the classification information of the label region of interest in the label data;

[0040] The eighth loss function is determined based on the difference between the location information of the second region of interest in the first target detection result and the location information of the label region of interest in the label data;

[0041] The first target loss function is constructed based on the fifth, sixth, seventh, and eighth loss functions.

[0042] In some embodiments, the second output unit includes:

[0043] The first enhancement subunit is used to weakly enhance the second image sample;

[0044] The first output subunit is used to input the weakly enhanced second image sample into the teacher detection model and output the corresponding second target detection result.

[0045] The second enhancement subunit is used to strongly enhance the second image sample;

[0046] The second output subunit is used to input the enhanced second image sample into the student detection model and output the corresponding third target detection result.

[0047] In some embodiments, the first output subunit is configured to:

[0048] The second image sample after weak enhancement is input into the teacher detection model, and the corresponding classification information of the third region, the location information of the third region, the classification information of the third region of interest, and the location information of the third region of interest are output.

[0049] The location information of the third region with a confidence level greater than the first preset threshold is determined as the location information of the third target region;

[0050] Determine the classification information of the third target region corresponding to the location information of the third target region;

[0051] The location information of the third region of interest with a confidence level greater than the second preset threshold is determined as the location information of the third target region of interest.

[0052] Determine the classification information of the third target's region of interest corresponding to its location information;

[0053] The classification information of the third target region, the location information of the third target region, the classification information of the third target region of interest, and the location information of the third target region of interest are determined as the second target detection result.

[0054] In some embodiments, the second output subunit is configured to:

[0055] The enhanced second image sample is input into the student detection model, which outputs the corresponding classification information of the fourth region, the location information of the fourth region, the classification information of the fourth region of interest, and the location information of the fourth region of interest.

[0056] The classification information of the fourth region, the location information of the fourth region, the classification information of the fourth region of interest, and the location information of the fourth region of interest are determined as the third target detection result.

[0057] In some embodiments, the second determining unit is configured to:

[0058] Determine the first target difference between the classification information of the third target region in the second target detection result and the classification information of the corresponding fourth region in the third target detection result;

[0059] Determine the second target difference between the location information of the third target region in the second target detection result and the location information of the corresponding fourth region in the third target detection result;

[0060] A ninth loss function is constructed based on the first target difference and the second target difference;

[0061] Determine the difference between the classification information of the third target region of interest in the second target detection result and the classification information of the corresponding fourth target region of interest in the third target detection result;

[0062] Determine the fourth target difference between the location information of the third target region of interest in the second target detection result and the location information of the corresponding fourth target region of interest in the third target detection result;

[0063] A tenth loss function is constructed based on the differences between the third and fourth objectives.

[0064] A second target loss function is constructed based on the ninth loss function and the tenth loss function.

[0065] In some embodiments, the iterative update unit is configured to:

[0066] The second network parameters of the teacher detection model are updated according to the exponential moving average method and the adjusted first network parameters.

[0067] The iteration returns to the point where the first image sample is input into the detection model, and the corresponding first detection result is output. This process is repeated to continuously update the parameters of the third network until the third objective loss function converges, thus obtaining the pre-trained detection model.

[0068] In some embodiments, the data processing apparatus includes:

[0069] The output unit is used to input the image to be labeled into the target detection model trained by the training method of the target detection model described above, and output the corresponding fourth target detection result.

[0070] The generation unit is used to generate target label information corresponding to the image to be labeled based on the fourth target detection result.

[0071] A computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform steps in the above-described object detection model training or data processing method.

[0072] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the above-described training or data processing method for the target detection model.

[0073] A computer program product or computer program includes computer instructions stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions, causing the computer to perform the steps in the above-described training or data processing method for the target detection model.

[0074] This embodiment of the application inputs a first image sample into a pre-trained detection model and outputs a corresponding first target detection result; determines a first target loss function based on the first target detection result and the label data carried by the first image sample; inputs a second image sample into a teacher detection model and a student detection model respectively, outputs corresponding second target detection results and third target detection results, and determines a second target loss function; updates the first network parameters of the student detection model based on the first and second target loss functions; and continuously updates the second network parameters of the teacher detection model based on the updated first network parameters until the iteration condition is met, thus stopping the training of the teacher detection model as the target detection model. This greatly improves the efficiency of data processing. In this way, a first image sample with labeled data is input into a pre-trained detection model to obtain a first target loss function with guiding characteristics. Second image samples without labeled data are input into the teacher detection model and the student detection model respectively, outputting corresponding second and third target detection results. A second target loss function is determined based on the difference between the second and third target detection results. The first and second target loss functions are combined to update the first network parameters of the student detection model, and the updated first network parameters guide the updating of the second network parameters of the teacher detection model. This process is iteratively repeated until the iteration condition is met, at which point the trained teacher detection model is identified as the target detection model. The target detection model, trained under the joint guidance of the first target loss function with labeled data and the second target loss function formed between the predictions of the teacher and student detection models, can output accurate detection results for automated labeling. Compared to schemes requiring manual labeling, this embodiment achieves efficient semi-supervised automatic training, obtaining a target detection model that can accurately achieve automatic labeling, reducing manual costs, and greatly improving the training efficiency of the target detection model. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a schematic diagram of a scenario for a training system of the target detection model provided in an embodiment of this application;

[0077] Figure 2a This is a flowchart illustrating the training method of the target detection model provided in the embodiments of this application;

[0078] Figure 2b This is a schematic diagram of the target detection network provided in an embodiment of this application;

[0079] Figure 2c A schematic diagram of a supervised training scenario provided for an embodiment of this application;

[0080] Figure 2d A schematic diagram illustrating an unsupervised training scenario provided in an embodiment of this application;

[0081] Figure 3 This is another flowchart illustrating the training method of the target detection model provided in the embodiments of this application;

[0082] Figure 4 This is a flowchart illustrating the data processing method provided in an embodiment of this application;

[0083] Figure 5 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application;

[0084] Figure 6 This is a schematic diagram of the server structure provided in the embodiments of this application. Detailed Implementation

[0085] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0086] This application provides a training method, apparatus, computer device, and storage medium for an object detection model. The training method for the object detection model can be applied to a training apparatus for the object detection model. The training apparatus can be integrated into a computer device, which can be a terminal with data processing capabilities. This terminal can be a smartphone, tablet, laptop, desktop computer, or smartwatch, but is not limited to these. The computer device can also be a server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0087] Please see Figure 1The figure shows a scenario diagram of the training system for the target detection model provided in this application. As shown, the computer device acquires a first image sample and a second image sample. The first image sample carries label data, while the second image sample does not. The first image sample is input into a pre-trained detection model, which outputs a corresponding first target detection result. A first target loss function is determined based on the first target detection result and the label data carried by the first image sample. The second image sample is input into a teacher detection model and a student detection model, respectively, and outputs corresponding second and third target detection results. A second target loss function is determined based on the difference between the second and third target detection results. The first network parameters of the student detection model are updated based on the first and second target loss functions. The second network parameters of the teacher detection model are updated based on the updated first network parameters, and the process iteratively returns to the previous step of inputting the first image sample into the pre-trained detection model and outputting the corresponding first target detection result, continuously updating the second network parameters until the iteration condition is met. The trained teacher detection model is then determined as the target detection model.

[0088] It should be noted that, Figure 1 The illustrated scenario of training the object detection model is merely an example. The training scenario of the object detection model described in the embodiments of this application is for the purpose of more clearly illustrating the technical solution of this application and does not constitute a limitation on the technical solution provided by this application. As those skilled in the art will understand, with the evolution of object detection model training and the emergence of new business scenarios, the technical solution provided by this application is also applicable to similar technical problems.

[0089] The following sections will provide detailed explanations.

[0090] In this embodiment, the description will be from the perspective of the training device for the target detection model, which can be integrated into a server that has storage units and is equipped with a microprocessor and has computing capabilities.

[0091] Please see Figure 2a , Figure 2a This is a flowchart illustrating the training method of the object detection model provided in this application embodiment. The training method of the object detection model includes:

[0092] In step 101, a first image sample and a second image sample are obtained.

[0093] Specifically, a first image sample and a second image sample can be obtained in advance. Both the first image sample and the second image sample are images containing a target object, which can be a person, an animal, or a building, etc. The number of images in the first image sample can be 1,000, and the number of images in the second image sample can be 5,000. This is an example and is not a specific limitation. The actual number can vary arbitrarily.

[0094] The first image sample carries label data, meaning that the target objects in the first image sample have been manually labeled in advance. This labeling method can include the location information of the target object in the corresponding area in the image and the classification information to which the target object belongs. The second image sample does not carry label data, meaning that the location information of the target object in the second image sample and the classification information to which the target object belongs are unknown.

[0095] In step 102, the first image sample is input into the pre-trained detection model, and the corresponding first target detection result is output.

[0096] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include information processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0097] The solutions provided in this application involve technologies such as artificial intelligence and computer vision, which are specifically illustrated through the following embodiments:

[0098] To better understand the embodiments of this application, it should be noted that the detection models, pre-trained detection models, teacher detection models, and student detection models in the embodiments of this application can all adopt the same object detection network architecture (Faster R-CNN). Please refer to these embodiments as well. Figure 2b As shown, the Figure 2b This is a schematic diagram of the target detection network provided in an embodiment of this application.

[0099] The object detection network 10 can be mainly divided into 4 parts:

[0100] The basic convolutional network 12 (Conv layers) is a convolutional neural network consisting of 13 convolutional (conv) layers, 13 linear rectified function (reLU) layers, and 4 pooling layers. It is mainly used to extract feature map information 13 from the image 11 to be processed.

[0101] Region Proposal Networks (RPNs) 14 are used to generate candidate regions. Specifically, they classify the anchors in the feature map information 13 using a normalization function (softmax), obtain positive and negative classification information, and determine the positive classification information as candidate regions (i.e., the location information of the regions corresponding to the regions in this embodiment). This preliminarily classifies the regions contained in the image corresponding to each object. The classification result corresponding to the candidate regions of the region proposal network is the classification information of the candidate regions of the region proposal network (i.e., the classification information of the regions corresponding to the regions in this embodiment), such as people, animals, or buildings.

[0102] Furthermore, the bounding box regression offset of the anchor can be calculated, and the candidate region can be adjusted according to the bounding box regression offset to obtain the final target candidate region 15 (proposal). At the same time, target candidate regions 15 that are too small or exceed the boundary are eliminated, thus realizing the positioning and selection of the target object.

[0103] The interest pooling layer 16 is responsible for collecting target candidate regions 15 and feature map information 13, and calculating the feature map information (proposal feature maps) of regions that meet the size conditions and sending them to subsequent layers for processing.

[0104] The classifier 17 can include a fully connected layer and a normalization layer. The classifier 17 combines the regional feature map information through the fully connected layer and the normalization layer to calculate the corresponding classification result of the regional feature map. At the same time, it can fine-tune the target candidate region 15 according to the classification result, and determine the fine-tuned target candidate region 15 as the final accurate detection region (i.e., the location information of the region of interest). The classification result corresponding to the location information of the region of interest is the classification information of the region of interest.

[0105] In practical applications, the cost of manually labeling data is extremely high. Therefore, the problem of how to automatically generate artificial labels for unlabeled data is an urgent issue to be solved.

[0106] While semi-supervised learning methods have made some progress in related technologies, enabling the prediction of artificially labeled data from unlabeled data, current semi-supervised learning methods are mostly applied to image classification and are difficult to apply to object detection technology. Object detection technology refers to scanning and searching for targets in images and videos (a series of images), which can be summarized as locating and identifying targets in a scene. This is mainly because current semi-supervised object detection methods have the following drawbacks in predicting artificially labeled data: 1. They do not utilize the network parameters of the trained detection model, resulting in insufficient accuracy of the initially generated artificially labeled data, thus affecting the training process of semi-supervised learning and leading to unfair results. 2. They do not use labeled data for learning guidance during the semi-supervised process, resulting in a certain gap between the predicted results and the actual results.

[0107] To address the aforementioned issues, embodiments of this application can pre-train the detection model iteratively based on first image samples carrying labeled data to obtain a pre-trained detection model. This training process can be as follows:

[0108] (1) Input the first image sample into the detection model and output the corresponding first detection result;

[0109] (2) Determine the third objective loss function based on the difference between the first detection result and the label data;

[0110] (3) Update the third network parameters of the detection model based on the third objective loss function, and iteratively return to execute the input of the first image sample into the detection model and output the corresponding first detection result, so as to continuously update the third network parameters until the third objective loss function converges and obtain the pre-trained detection model.

[0111] (4) Pass the first target network parameters of the pre-trained detection model to the teacher detection model and the student detection model.

[0112] The first image sample is input into the detection model, which outputs a first detection result based on the first image sample to predict the target object in the corresponding image. The first detection result may include the location information of the first region, the classification information of the first region, the location information of the first region of interest, and the classification information of the first region of interest.

[0113] Furthermore, since the label data represents the actual detection results corresponding to the first image sample, a third objective loss function can be determined based on the difference between the first detection result and the label data. The loss value calculated by the loss function reflects the difference between the predicted detection result and the label data. By minimizing the loss function, the model can reach convergence, achieving more accurate prediction capabilities. Therefore, the larger the third loss value calculated by the third objective loss function, the greater the difference between the first detection result and the label data, and the less accurate the prediction. Conversely, the smaller the third loss value, the smaller the difference between the first detection result and the label data, and the more accurate the prediction.

[0114] Correspondingly, the third network parameters of the first detection model can be back-derived according to the third objective loss function to update the third network parameters. The detection model after the third network parameters are updated by continuously inputting the first image sample into the model can be returned, and a new first detection result can be output. Based on the difference between the new first detection result and the label data, a new third objective loss function can be determined, and the third network parameters of the detection model can be updated again. As the iteration process proceeds, the value of the third objective loss function will become smaller and smaller until it converges to a certain value, which means that the detection model has been trained and a pre-trained detection model is obtained. The pre-trained detection model has good prediction ability after iterative training with the first image sample with label data and can output a relatively accurate prediction result for the target object. The pre-trained detection model carries the first objective network parameters.

[0115] In order to achieve subsequent unsupervised learning, the first target network parameters of the pre-trained detection model can be passed to the teacher detection model and the student detection model. That is, the initial network parameters of the teacher detection model and the student detection model can be directly set to the trained first target network parameters, so that the initial prediction ability of the teacher detection model and the student detection model can be guaranteed.

[0116] In some implementations, determining a third objective loss function based on the difference between the first detection result and the label data may include:

[0117] (1.1) Determine the first loss function based on the difference between the classification information of the first region in the first detection result and the classification information of the label region in the label data;

[0118] (1.2) Determine the second loss function based on the difference between the location information of the first region in the first detection result and the location information of the label region in the label data;

[0119] (1.3) Determine the third loss function based on the difference between the classification information of the first region of interest in the first detection result and the classification information of the label region of interest in the label data;

[0120] (1.4) Determine the fourth loss function based on the difference between the location information of the first region of interest in the first detection result and the location information of the label region of interest in the label data;

[0121] (1.5) Construct a third objective loss function based on the first loss function, the second loss function, the third loss function and the fourth loss function.

[0122] The label data can store pre-annotated, accurate classification information of label regions, location information of label regions, classification information of label regions of interest, and location information of label regions of interest. In this embodiment, the purpose of training the detection model is to enable it to output predictions close to the label data based on the first sample image. Therefore, a first loss function can be determined based on the similarity between the classification information of the first region in the first detection result and the classification information of the label regions in the label data. A second loss function is determined based on the degree of overlap between the location information of the first region in the first detection result and the location information of the label regions in the label data.

[0123] A third loss function is determined based on the similarity between the classification information of the first region of interest in the first detection result and the classification information of the label region of interest in the label data. A fourth loss function is determined based on the degree of overlap between the location information of the first region of interest in the first detection result and the location information of the label region of interest in the label data.

[0124] Furthermore, a third objective loss function can be constructed based on the first, second, third, and fourth loss functions. For example, the third objective loss function can be constructed using the following formula:

[0125]

[0126] The L S This is the third objective loss function, the RPN classification loss function. (That is, the first loss function), RPN regression loss function (This is the second loss function), ROI classification loss function (That is, the third loss function), ROI regression loss function (That is, the fourth loss function), and thus, the four loss functions can be added together to construct the third target loss function for training.

[0127] Based on this, the first image sample with labeled data can be input into the pre-trained detection model, which will output the corresponding first target detection result. Please refer to the following: Figure 2c As shown, Figure 2c This is a schematic diagram of a supervised training scenario provided in an embodiment of this application. In this supervised training scenario 20, the pre-trained detection model can predict the classification information and location information of the second region corresponding to the RPN, as well as the classification information and location information of the second region of interest corresponding to the ROI, for the first image sample. The classification information, location information, and location information of the second region can be used as the first target detection result. In step 103, a first target loss function is determined based on the first target detection result and the label data carried by the first image sample.

[0128] This step can be understood as the supervised learning process in semi-supervised learning, that is, the difference between the first target detection result and the label data can be used to determine the first target loss function (Loss1), which is then used to guide the unsupervised learning process.

[0129] In some implementations, the step of determining the first target loss function based on the first target detection result and the label data carried by the first image sample includes:

[0130] (1) Determine the fifth loss function based on the difference between the classification information of the second region in the first target detection result and the classification information of the label region in the label data;

[0131] (2) Determine the sixth loss function based on the difference between the location information of the second region in the first target detection result and the location information of the label region in the label data;

[0132] (3) Determine the seventh loss function based on the difference between the classification information of the second region of interest in the first target detection result and the classification information of the label region of interest in the label data;

[0133] (4) Determine the eighth loss function based on the difference between the location information of the second region of interest in the first target detection result and the location information of the label region of interest in the label data;

[0134] (5) Construct the first target loss function based on the fifth, sixth, seventh and eighth loss functions.

[0135] The label data can store pre-annotated, accurate classification information of label regions, location information of label regions, classification information of label regions of interest, and location information of label regions of interest. Therefore, a fifth loss function can be determined based on the similarity between the classification information of the second region in the first target detection result and the classification information of the label regions in the label data. A sixth loss function is determined based on the degree of overlap between the classification information of the second region in the first target detection result and the classification information of the label regions in the label data.

[0136] The seventh loss function is determined based on the similarity between the classification information of the second region of interest in the first object detection result and the classification information of the label region of interest in the label data. The eighth loss function is determined based on the degree of overlap between the location information of the second region of interest in the first object detection result and the location information of the label region of interest in the label data.

[0137] The first objective loss function is constructed based on the fifth, sixth, seventh, and eighth loss functions.

[0138] In step 104, the second image sample is input into the teacher detection model and the student detection model respectively, and the corresponding second target detection result and third target detection result are output.

[0139] The teacher detection model has a complex network structure and a large number of parameters, resulting in high recognition accuracy. The student detection model has a simpler network structure and a smaller number of parameters, therefore its accuracy is slightly lower than the teacher detection model.

[0140] This step can be understood as an unsupervised learning process within semi-supervised learning. To achieve automatic generation of subsequent labeled data, second image samples without labeled data can be input into the teacher detection model and the student detection model respectively, outputting the corresponding second and third object detection results. Please refer to the following: Figure 2d As shown, Figure 2d This is a schematic diagram of an unsupervised training scenario provided in an embodiment of this application. In this unsupervised training scenario 30, the classification information and location information of the third region corresponding to the RPN, as well as the classification information and location information of the third region of interest corresponding to the ROI, can be obtained by the teacher detection model from the second image sample. The classification information, location information, and location information of the third region can be used as the second target detection result. The second target detection result can also be understood as a pseudo-label, which can be expressed in a one-hot encoding manner.

[0141] To verify the pseudo-label, the student detection model can be used to predict the classification information and location information of the fourth region corresponding to the RPN, as well as the classification information and location information of the fourth region of interest corresponding to the ROI, from the second image sample. The classification information, location information, and location information of the fourth region can be used as the third target detection result. This third target detection result can also be understood as the prediction result (data distribution), which is used to compare with the second target detection result.

[0142] It should be noted that the network parameters of both the initial teacher detection model and the student detection model are the first target network parameters that have been trained and passed on from the first target detection model. As a result, the accuracy of the pseudo-labels output by the teacher detection model and the data distribution of the student model are greatly improved, making the output results more fair and effective.

[0143] In some implementations, the second image sample is input into the teacher detection model and the student detection model respectively, and the corresponding second target detection result and third target detection result are output, which may include:

[0144] (1) Perform weak enhancement on the second image sample;

[0145] (2) Input the second image sample after weak enhancement into the teacher detection model and output the corresponding second target detection result;

[0146] (3) Enhance the second image sample;

[0147] (4) Input the enhanced second image sample into the student detection model and output the corresponding third target detection result.

[0148] The weak enhancement can be achieved by flipping and translating, while the strong enhancement can be achieved by occluding part of the image. The above enhancement methods can enhance the robustness of the finally trained model. Thus, the second image sample can be weakly enhanced, and the weakly enhanced second image sample can be input into the teacher detection model. The output of the third region generation network classification information, the third region generation network candidate region information, the third region of interest classification information, and the third region of interest location information are used as the second target detection result (pseudo-label).

[0149] The second image sample is strongly enhanced, and the enhanced second image sample is input into the student detection model. The output of the fourth region generation network classification information, the fourth region generation network candidate region information, the fourth region of interest classification information, and the fourth region of interest location information are used as the third target detection result (i.e., the prediction result).

[0150] In step 105, a second target loss function is determined based on the difference between the second target detection result and the third target detection result.

[0151] In order to achieve unsupervised learning, it is necessary to verify the accuracy of the second target detection result (i.e., pseudo-label) output by the teacher detection model. However, since the input second image sample is an image of unlabeled data, the pseudo-label cannot be directly verified. In this embodiment, the third target detection result can be used as the prediction result and compared with the pseudo-label. The second target loss function (Loss2) is determined based on the difference between the two. The smaller the second target loss function, the closer the prediction result and the predicted pseudo-label are, and the more accurate the prediction result and the predicted pseudo-label are. The larger the second target loss function, the less close the prediction result and the predicted pseudo-label are, and the less accurate the prediction result and the predicted pseudo-label are.

[0152] In step 106, the first network parameters of the student detection model are updated based on the first objective loss function and the second objective loss function.

[0153] In contrast to related technologies, the embodiments of this application can combine the first objective loss function derived from labeled data in supervised learning and the second objective loss function between pseudo-labels and prediction results in unsupervised learning to perform inverse differentiation on the first network parameters of the student detection model, so as to update the first network parameters and make the prediction of the second image sample with unlabeled data more and more accurate.

[0154] In step 107, the second network parameters of the teacher detection model are updated according to the updated first network parameters, and the process iteratively returns to input the first image sample into the pre-trained detection model and output the corresponding first target detection result, so as to continuously update the second network parameters until the iteration condition is met and the trained teacher detection model is determined as the target detection model.

[0155] The second network parameters of the teacher detection model can be adjusted and updated in real time based on the adjusted first network parameters. This can be done by directly replacing the second network parameters with the first network parameters, or by using an exponential moving average (EMA) to update the second network parameters based on the first network parameters. The EMA is used to average the model's parameters to improve the prediction accuracy and robustness of the teacher detection model. For example, please refer to the following formula:

[0156] Θ t =αΘ t +(1-α)Θ s

[0157] The Θ t For the second network parameter, Θ s The first network parameter is α, which is a hyperparameter used to adjust the weights. The second network parameter of the teacher detection model is then updated using the above formula, so that the accuracy of the teacher detection model's predictions becomes increasingly higher.

[0158] Furthermore, the process continuously returns to the pre-trained detection model, inputting the first image sample, outputting the corresponding first target detection result, and recalculating the new first and second target loss functions to continuously update the first and second network parameters. The specific update method is detailed in the steps above. It's important to note that after each iteration, the first and second network parameters of both the teacher and student detection models change, resulting in continuous changes in the second and third target detection results. The iteration condition can be a preset number of iterations, such as 100 or 500. When the preset number of iterations is reached, training stops, and the last trained teacher detection model is designated as the target detection model. Due to continuous optimization, the second network parameters of this target detection model are updated to parameters close to the actual values. Therefore, this target teacher detection model can be used to predict images with unlabeled data, and the output results can be directly used as the label data for the predicted images. This label data can include the classification information and location information of the predicted region corresponding to the RPN output by the target detection model, as well as the classification information and location information of the predicted region of interest corresponding to the ROI.

[0159] As described above, this embodiment of the application inputs a first image sample into a pre-trained detection model and outputs a corresponding first target detection result; determines a first target loss function based on the first target detection result and the label data carried by the first image sample; inputs a second image sample into both the teacher detection model and the student detection model, outputting corresponding second and third target detection results to determine a second target loss function; updates the first network parameters of the student detection model based on the first and second target loss functions; updates the second network parameters of the teacher detection model based on the updated first network parameters, and so on, continuously updating the second network parameters until the iteration condition is met, thus determining the trained teacher detection model as the target detection model. This greatly improves the efficiency of data processing. In this way, a first image sample with labeled data is input into a pre-trained detection model to obtain a first target loss function with guiding characteristics. Second image samples without labeled data are input into the teacher detection model and the student detection model respectively, outputting corresponding second and third target detection results. A second target loss function is determined based on the difference between the second and third target detection results. The first and second target loss functions are combined to update the first network parameters of the student detection model, and the updated first network parameters guide the updating of the second network parameters of the teacher detection model. This process is iteratively repeated until the iteration condition is met, at which point the trained teacher detection model is identified as the target detection model. The target detection model, trained under the joint guidance of the first target loss function with labeled data and the second target loss function formed between the predictions of the teacher and student detection models, can output accurate detection results for automated labeling. Compared to schemes requiring manual labeling, this embodiment achieves efficient semi-supervised automatic training, obtaining a target detection model that can accurately achieve automatic labeling, reducing manual costs, and greatly improving the training efficiency of the target detection model.

[0160] In this embodiment, the training device for the target detection model will be specifically integrated into the server as an example for explanation. Please refer to the following description for details.

[0161] Please see Figure 3 , Figure 3 Another flowchart illustrating the training method for the object detection model provided in this application embodiment. The method flow may include:

[0162] In step 201, the server acquires the first image sample and the second image sample.

[0163] Specifically, 1,000 first image samples and 3,000 second image samples can be pre-acquired. Both the first and second image samples contain images of animal objects. The first image samples carry label data, meaning that both animal objects and other non-animal objects in the first image samples are pre-labeled manually. The labeling method can be to use a label box to mark the area information occupied by each object in the image and record the classification information corresponding to the object. The classification information is the category of the object, such as cat, dog, or human, etc. The area information can include the position information and size information of the label box.

[0164] The second image sample does not carry label data, meaning that the information about the area occupied by the object in the second image sample and the classification information to which the target object belongs are completely unknown.

[0165] The first image sample can be expressed by the formula This indicates that the D S This is the first image sample, N. S This represents the number of samples in the first image. Representing the i-th first image sample, This represents the corresponding tag data.

[0166] The second image sample can be expressed by the formula This indicates that the D U This is the second image sample, N. U This represents the number of second image samples. This represents the i-th second image sample.

[0167] To meet the subsequent training conditions, the label data can include the classification information of the label region, the location information of the label region, the classification information of the label region of interest, and the location information of the label region of interest. The location information of the label region is the region information of all objects in the first image sample, and the classification information of the label region is the classification information corresponding to the location information of each label region.

[0168] The location information of the region of interest for the label is the region information of the object of interest that we ultimately use for classification in the first image sample. For example, if the object detection task is to distinguish between animals, then cats and dogs are the objects of interest relative to cats, dogs and humans. Therefore, the region information of cats and dogs is the location information of the region of interest for the label. The classification information of the region of interest for the label is the classification result corresponding to the location information of the region of interest for each label.

[0169] In step 202, the server inputs the first image sample into the detection model and outputs the corresponding first detection result. A first loss function is determined based on the difference between the classification information of the first region in the first detection result and the classification information of the label region in the label data. A second loss function is determined based on the difference between the location information of the first region in the first detection result and the location information of the label region in the label data.

[0170] In step 203, the server determines a third loss function based on the difference between the classification information of the first region of interest in the first detection result and the classification information of the label region of interest in the label data, and determines a fourth loss function based on the difference between the location information of the first region of interest in the first detection result and the location information of the label region of interest in the label data. The server then constructs a third target loss function based on the first loss function, the second loss function, the third loss function, and the fourth loss function.

[0171] It should be noted that the detection models, pre-trained detection models, teacher detection models, and student detection models in the embodiments of this application can all adopt the same target detection network architecture.

[0172] The server inputs the first image sample into the detection model, which outputs a first detection result based on the first image sample to predict the target object in the corresponding image. The first detection result may include the classification information and location information of the first region corresponding to the RPN, as well as the classification information and location information of the first region of interest corresponding to the ROI.

[0173] Furthermore, since the label data represents the actual detection results corresponding to the first image sample, a first loss function can be determined based on the distributional similarity between the classification information of the first region in the first detection result and the classification information of the label region in the label data. A second loss function is determined based on the image spatial overlap between the location information of the first region in the first detection result and the location information of the label region in the label data.

[0174] A third loss function is determined based on the distributional similarity between the classification information of the first region of interest in the first detection result and the classification information of the labeled regions of interest in the label data. A fourth loss function is determined based on the image spatial overlap between the location information of the first region of interest in the first detection result and the location information of the labeled regions of interest in the label data.

[0175] Furthermore, a third objective loss function can be constructed based on the first, second, third, and fourth loss functions. For example, the third objective loss function can be constructed using the following formula:

[0176]

[0177] The L S This is the third objective loss function, the RPN classification loss function. (That is, the first loss function), RPN regression loss function (This is the second loss function), ROI classification loss function (That is, the third loss function), ROI regression loss function (That is, the fourth loss function), and thus, the four loss functions can be added together to construct the third target loss function for training.

[0178] In step 204, the server updates the third network parameters of the detection model based on the third objective loss function, and iteratively returns to execute the input of the first image sample into the detection model and output the corresponding first detection result, so as to continuously update the third network parameters until the third objective loss function converges, thereby obtaining the pre-trained detection model.

[0179] Correspondingly, the third network parameters of the detection model can be back-derived according to the third objective loss function to update the third network parameters. Steps 202, 203, and 204 are continuously returned to input the first image sample into the detection model after the third network parameters are updated, and a new first detection result is output. Based on the difference between the new first detection result and the label data, a new third objective loss function is determined, and the third network parameters of the detection model are updated. As the iteration process proceeds, the value of the third objective loss function will become smaller and smaller until it converges to a certain value, which means that the detection model has been trained and a pre-trained detection model is obtained. The pre-trained detection model has good target prediction ability after iterative training with the first image sample with label data and can output a relatively accurate prediction result of the target object. The pre-trained detection model carries the first objective network parameters.

[0180] In step 205, the server passes the first target network parameters of the pre-trained detection model to the teacher detection model and the student detection model.

[0181] In order to achieve subsequent unsupervised learning, the first target network parameter Θ of the pre-trained detection model can be passed to the teacher detection model and the student detection model. That is, the initial network parameters of the teacher detection model and the student detection model can be directly set to the trained first target network parameters, so that the initial prediction ability of the second target detection model, the teacher detection module and the student detection model can be guaranteed.

[0182] In step 206, the server inputs the first image sample into the pre-trained detection model and outputs the corresponding first target detection result. The fifth loss function is determined based on the difference between the classification information of the second region in the first target detection result and the classification information of the label region in the label data. The sixth loss function is determined based on the difference between the location information of the second region in the first target detection result and the location information of the label region in the label data.

[0183] In step 207, the server determines a seventh loss function based on the difference between the classification information of the second region of interest in the first target detection result and the classification information of the label region of interest in the label data, determines an eighth loss function based on the difference between the location information of the second region of interest in the first target detection result and the location information of the label region of interest in the label data, and constructs a first target loss function based on the fifth, sixth, seventh and eighth loss functions.

[0184] This step can be understood as the supervised learning process in semi-supervised learning. The first image sample and the second image sample are mixed together and randomly shuffled. A certain number of image samples are imported each time according to the batch size. The image samples can include the randomly mixed first image sample and the second image sample. The batch size can be 500. It should be noted that after each batch of image samples is trained, the next batch of image samples will be automatically imported.

[0185] For a random mixture of 500 image samples, the first image sample with labeled data is predicted by a pre-trained detection model to obtain the classification information and location information of the second region corresponding to the RPN, as well as the classification information and location information of the second region of interest corresponding to the ROI. The classification information, location information, and location information of the second region are used as the first target detection result.

[0186] The label data can store pre-annotated, accurate classification information of label regions, location information of label regions, classification information of label regions of interest, and location information of label regions of interest. Therefore, a fifth loss function can be determined based on the distributional similarity between the classification information of the second region in the first target detection result and the classification information of the label regions in the label data. A sixth loss function is determined based on the image spatial overlap between the location information of the second region in the first target detection result and the location information of the label regions in the label data.

[0187] The seventh loss function is determined based on the distributional similarity between the classification information of the second region of interest in the first object detection result and the classification information of the labeled region of interest in the label data. The eighth loss function is determined based on the image spatial overlap between the location information of the second region of interest in the first object detection result and the location information of the labeled region of interest in the label data.

[0188] The first objective loss function, Loss1, is constructed based on the fifth, sixth, seventh, and eighth loss functions.

[0189] In step 208, the server performs weak enhancement on the second image sample, inputs the weakly enhanced second image sample into the teacher detection model, and outputs the classification information of the third region, the location information of the third region, the classification information of the third region of interest, and the location information of the third region of interest. The location information of the third region with a confidence level greater than the first preset threshold is determined as the location information of the third target region, and the classification information of the third target region corresponding to the location information of the third target region is determined.

[0190] This step can be understood as an unsupervised learning process within semi-supervised learning. To achieve automatic generation of subsequent label data, the second image samples without label data from a randomly mixed pool of 500 image samples undergo a weak enhancement operation involving flipping and translation. These samples are then input into the teacher detection model, which outputs the classification and location information of the third region corresponding to the RPN, as well as the classification and location information of the third region of interest corresponding to the ROI.

[0191] It should be noted that each location information of the third region corresponds to a certain confidence level, which ranges from (0, 1). The higher the confidence level, the greater the probability of predicting it as the classification information of the corresponding third region; the lower the confidence level, the lower the probability of predicting it as the classification information of the corresponding third region. To achieve better training results, this embodiment can set a first preset threshold, which can be 0.3. Therefore, the location information of the third region with a confidence level greater than 0.3 is determined as the location information of the third target region, that is, only the location information of the third region with a confidence level greater than the first preset threshold is retained.

[0192] Accordingly, the classification information of the third region corresponding to the location information of the third target region is determined as the classification information of the third target region generation network, and the location information of the remaining third regions that are less than the first preset threshold are not trained.

[0193] In step 209, the server determines the location information of the third region of interest with a confidence level greater than the second preset threshold as the location information of the third target region of interest, determines the classification information of the third target region of interest corresponding to the location information of the third target region of interest, and determines the classification information of the third target region, the location information of the third target region, the classification information of the third target region of interest, and the location information of the third target region of interest as the second target detection result.

[0194] Each region of interest (ROI) has a corresponding confidence level, ranging from 0 to 1. A higher confidence level indicates a greater probability of being predicted as a ROI, while a lower confidence level indicates a lower probability. To achieve better training results, this embodiment can set a second preset threshold, where the first preset threshold can be 0.4. This allows the location information of ROIs with a confidence level greater than 0.4 to be identified as the location information of the third target ROI, meaning only the location information of ROIs with a confidence level greater than the second preset threshold is retained.

[0195] Accordingly, the classification information of the third region of interest corresponding to the location information of the third target region of interest is determined as the classification information of the third target region of interest, and the location information of the remaining third regions of interest that are less than the second preset threshold are not trained.

[0196] Finally, the classification information of the filtered third target region, the location information of the third target region, the classification information of the third target region of interest, and the location information of the third target region of interest are determined as the second target detection result (i.e., pseudo-labels, which can be understood as temporary label data of the second image sample).

[0197] In step 210, the server performs strong enhancement on the second image sample, inputs the enhanced second image sample into the student detection model, and outputs the corresponding classification information of the fourth region, the location information of the fourth region, the classification information of the fourth region of interest, and the location information of the fourth region of interest. The classification information of the fourth region, the location information of the fourth region, the classification information of the fourth region of interest, and the location information of the fourth region of interest are determined as the third target detection result.

[0198] Specifically, the second image sample without label data from a random mix of 500 image samples can be subjected to a strong enhancement operation that randomly occludes part of the image region. This enhanced image sample is then input into the student detection model, which outputs the classification information and location information of the fourth region corresponding to the RPN, as well as the classification information and location information of the fourth region of interest corresponding to the ROI. The classification information, location information, and location information of the fourth region are then used as the third target detection result (i.e., the prediction result, which can be understood as the prediction result of the second image sample).

[0199] In step 211, the server determines the first target difference between the classification information of the third target region in the second target detection result and the classification information of the corresponding fourth region in the third target detection result, determines the second target difference between the location information of the third target region in the second target detection result and the location information of the corresponding fourth region in the third target detection result, and constructs a ninth loss function based on the first target difference and the second target difference.

[0200] Please refer to the following formulas as well:

[0201]

[0202] Should This represents the ninth loss function. and This represents the classification information of the third target region and the location information of the third target region corresponding to all classes in the teacher detection model. and This represents the classification information of the fourth region in the student detection model and the location information of the fourth region corresponding to all classes. This D... KL Let KL divergence represent the difference between the distributions of two classification information. The greater the difference, the greater the relative entropy; the smaller the difference, the smaller the relative entropy. Based on this, the KL divergence between the classification information of the third target region and the classification information of the fourth region is calculated using the above formula to determine the first target difference. The second target difference is calculated between the location information of the third target region and the location information of the corresponding fourth region in the third target detection result. The ninth loss is constructed based on the sum of these two differences.

[0203] In step 212, the server determines the third target difference between the classification information of the third target region of interest in the second target detection result and the classification information of the corresponding fourth target region of interest in the third target detection result, determines the fourth target difference between the location information of the third target region of interest in the second target detection result and the location information of the corresponding fourth target region of interest in the third target detection result, constructs a tenth loss function based on the third target difference and the fourth target difference, and constructs a second target loss function based on the ninth loss function and the tenth loss function.

[0204] Please refer to the following formulas as well:

[0205]

[0206] Should This represents the tenth loss function. and This represents the classification information of the third target region of interest predicted by the teacher detection model and the location information of the third target region of interest corresponding to each classification. and This represents the classification information of the fourth region of interest (ROI) in the student detection model and the location information of the fourth ROI for each class. This D... KL Let KL divergence be represented. Using the above formula, the KL divergence between the classification information of the third target region of interest and the classification information of the fourth target region of interest is calculated to determine the difference of the third target. The difference of the fourth target is then calculated between the location information of the third target region of interest and the location information of the corresponding fourth target region of interest in the third target detection result. The tenth loss is constructed based on the sum of these two values.

[0207] Please refer to the following formula:

[0208]

[0209] Loss2 is the second objective loss function, which can be constructed by summing the ninth and tenth loss functions.

[0210] In step 213, the server updates the first network parameters of the student detection model based on the first objective loss function and the second objective loss function.

[0211] Please refer to the following formulas as well:

[0212] L = Loss1 + λLoss2, where λ is a hyperparameter that determines the contribution of the loss corresponding to the unlabeled second image sample to the labeled first image sample. In one embodiment, L is the fourth objective loss function. Thus, the second objective loss function can be weighted by λ to obtain the weighted second objective loss function. The fourth objective loss function is then constructed based on the first objective loss function and the weighted second objective loss function.

[0213] Furthermore, the first network parameters Θ of the student detection model can be adjusted based on the fourth objective loss function L. s Perform inverse differentiation to obtain the first network parameter Θ s The system is updated to make the student detection model increasingly accurate in predicting second image samples from unlabeled data. Because the four-objective loss function L uses guidance from the real labeled data in Loss1, both the student and teacher detection models will become more accurate during training.

[0214] In step 214, the server updates the second network parameters of the teacher detection model according to the exponential moving average method and the adjusted first network parameters, and iteratively returns to the process of inputting the first image sample into the pre-trained detection model and outputting the corresponding first target detection result, so as to continuously update the second network parameters until the iteration condition is met and the trained teacher detection model is determined as the target detection model.

[0215] This can be achieved through exponential moving average, and based on the adjusted first network parameter Θ. s Update the second network parameter Θ of the teacher detection model. t Exponential moving average is used to average the parameters of a model in order to improve the teacher evaluation metrics of the model and increase its robustness. For example, please refer to the following formula:

[0216] Θ t =αΘ t +(1-α)Θ s

[0217] The Θ t For the second network parameter, Θ s The first network parameter is α, which is a hyperparameter used to adjust the weights. The second network parameter of the teacher detection model is then updated using the above formula, so that the accuracy of the teacher detection model's predictions becomes increasingly higher.

[0218] Furthermore, the process continuously returns to steps 206 to 214 to continuously input the first image sample into the pre-trained detection model, output the corresponding first target detection result, and recalculate the new first target loss function and second target loss function to continuously update the first and second network parameters. For specific update methods, please refer to the steps above. It should be noted that after each iteration, the first and second network parameters of both the teacher detection model and the student detection model will change, so the second and third target detection results will also change continuously. The iteration condition can be a preset number of iterations, such as 100 or 500 times, etc. When the number of iterations is detected to have reached the preset number of iterations, training stops, and the teacher detection model trained last time is determined as the target detection model. Due to continuous optimization, the second network parameters of the target detection model are also updated to parameters close to the real ones. Therefore, the target detection model can be used to predict the labels of unlabeled images.

[0219] As described above, this embodiment of the application inputs a first image sample into a pre-trained detection model and outputs a corresponding first target detection result; determines a first target loss function based on the first target detection result and the label data carried by the first image sample; inputs a second image sample into both a teacher detection model and a student detection model, outputting corresponding second and third target detection results to determine a second target loss function; updates the first network parameters of the student detection model based on the first and second target loss functions; and continuously updates the second network parameters of the teacher detection model based on the updated first network parameters until the iteration condition is met, thus determining the trained teacher detection model as the target detection model. This significantly improves the efficiency of data processing. In this way, a first image sample with labeled data is input into a pre-trained detection model to obtain a first target loss function with guiding characteristics. Second image samples without labeled data are input into the teacher detection model and the student detection model respectively, outputting corresponding second and third target detection results. A second target loss function is determined based on the difference between the second and third target detection results. The first and second target loss functions are combined to update the first network parameters of the student detection model, and the updated first network parameters guide the updating of the second network parameters of the teacher detection model. This process is iteratively repeated until the iteration condition is met, at which point the trained teacher detection model is identified as the target detection model. The target detection model, trained under the joint guidance of the first target loss function with labeled data and the second target loss function formed between the predictions of the teacher and student detection models, can output accurate detection results for automated labeling. Compared to schemes requiring manual labeling, this embodiment achieves efficient semi-supervised automatic training, obtaining a target detection model that can accurately achieve automatic labeling, reducing manual costs, and greatly improving the training efficiency of the target detection model.

[0220] In this embodiment, the description will be from the perspective of a data processing device, which can be integrated into a server that has storage units and is equipped with a microprocessor and has computing capabilities.

[0221] Please refer to 4. Figure 4 This is a flowchart illustrating the data processing method provided in an embodiment of this application. The data processing method includes:

[0222] In step 215, the image to be labeled is input into the target detection model trained by the above target detection model training method, and the corresponding fourth target detection result is output.

[0223] In this method, unlabeled images to be labeled can be input into the target detection model trained using the training method described above. This target detection model can directly output the classification information and location information of the predicted region corresponding to the RPN, as well as the classification information and location information of the predicted region of interest corresponding to the ROI. The classification information, location information, and location information of the predicted region are used as the fourth target detection result. Since the initial network parameters of this target detection model are the first target network parameters determined by the pre-trained detection model based on the label data, and the network parameters are further guided by the label data in subsequent training, the fourth target detection result predicted by the target detection model is fair and accurate.

[0224] In step 216, target label information corresponding to the image to be labeled is generated based on the fourth target detection result.

[0225] The fourth object detection result can be directly used as the target label information. Alternatively, the classification information and location information of the predicted region of interest can be used as label data, enabling automatic label data generation in the future. This significantly reduces costs and improves data processing efficiency.

[0226] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a training device for a target detection model provided in an embodiment of this application. The training device for the target detection model may include an acquisition unit 301, a first output unit 302, a first determination unit 303, a second output unit 304, a second determination unit 305, a first update unit 306, and an iterative update unit 307, etc.

[0227] The acquisition unit 301 is used to acquire a first image sample and a second image sample, wherein the first image sample carries label data and the second image sample does not carry label data.

[0228] In some embodiments, the device further includes:

[0229] The third output unit is used to input the first image sample into the detection model and output the corresponding first detection result.

[0230] The third determining unit is used to determine a third target loss function based on the difference between the first detection result and the label data;

[0231] The second update unit is used to update the third network parameters of the detection model based on the third objective loss function, and iteratively return to execute the input of the first image sample into the detection model and output the corresponding first detection result, so as to continuously update the third network parameters until the third objective loss function converges and the pre-trained detection model is obtained.

[0232] The transmission unit is used to pass the first target network parameters of the pre-trained detection model to the teacher detection model and the student detection model.

[0233] In some embodiments, the third determining unit is configured to:

[0234] The first loss function is determined based on the difference between the classification information of the first region in the first detection result and the classification information of the label region in the label data;

[0235] The second loss function is determined based on the difference between the location information of the first region in the first detection result and the location information of the label region in the label data;

[0236] The third loss function is determined based on the difference between the classification information of the first region of interest in the first detection result and the classification information of the label region of interest in the label data;

[0237] The fourth loss function is determined based on the difference between the location information of the first region of interest in the first detection result and the location information of the label region of interest in the label data;

[0238] A third objective loss function is constructed based on the first, second, third, and fourth loss functions.

[0239] The first output unit 302 is used to input the first image sample into the pre-trained detection model and output the corresponding first target detection result.

[0240] The first determining unit 303 is used to determine a first target loss function based on the first target detection result and the label data carried by the first image sample.

[0241] In some embodiments, the first determining unit 303 is configured to:

[0242] The fifth loss function is determined based on the difference between the classification information of the second region in the first target detection result and the classification information of the label region in the label data;

[0243] The sixth loss function is determined based on the difference between the location information of the second region in the first target detection result and the location information of the label region in the label data;

[0244] The seventh loss function is determined based on the difference between the classification information of the second region of interest in the first target detection result and the classification information of the label region of interest in the label data;

[0245] The eighth loss function is determined based on the difference between the location information of the second region of interest in the first target detection result and the location information of the label region of interest in the label data;

[0246] The first objective loss function is constructed based on the fifth, sixth, seventh, and eighth loss functions.

[0247] The second output unit 304 is used to input the second image sample into the teacher detection model and the student detection model respectively, and output the corresponding second target detection result and the third target detection result.

[0248] In some embodiments, the second output unit 304 includes:

[0249] The first enhancement subunit is used to weakly enhance the second image sample;

[0250] The first output subunit is used to input the weakly enhanced second image sample into the teacher detection model and output the corresponding second target detection result.

[0251] The second enhancement subunit is used to strongly enhance the second image sample;

[0252] The second output subunit is used to input the enhanced second image sample into the student detection model and output the corresponding third target detection result.

[0253] In some embodiments, the first output subunit is used for:

[0254] The second image sample after weak enhancement is input into the teacher detection model, and the corresponding classification information of the third region, the location information of the third region, the classification information of the third region of interest, and the location information of the third region of interest are output.

[0255] The location information of the third region with a confidence level greater than the first preset threshold is determined as the location information of the third target region;

[0256] Determine the classification information of the third target region corresponding to its location information;

[0257] The location information of the third region of interest with a confidence level greater than the second preset threshold is determined as the location information of the third target region of interest.

[0258] Determine the classification information of the third target's region of interest corresponding to its location information;

[0259] The classification information of the third target region, the location information of the third target region, the classification information of the third target region of interest, and the location information of the third target region of interest are determined as the second target detection result.

[0260] In some embodiments, the second output subunit is used for:

[0261] The enhanced second image sample is input into the student detection model, which outputs the corresponding classification information of the fourth region, the location information of the fourth region, the classification information of the fourth region of interest, and the location information of the fourth region of interest.

[0262] The classification information of the fourth region, the location information of the fourth region, the classification information of the fourth region of interest, and the location information of the fourth region of interest are determined as the third target detection result.

[0263] The second determining unit 305 is used to determine the second target loss function based on the difference between the second target detection result and the third target detection result.

[0264] In some embodiments, the second determining unit 305 is configured to:

[0265] Determine the first target difference between the classification information of the third target region in the second target detection result and the classification information of the corresponding fourth region in the third target detection result;

[0266] Determine the difference between the location information of the third target region in the second target detection result and the location information of the corresponding fourth region in the third target detection result;

[0267] A ninth loss function is constructed based on the difference between the first objective and the difference between the second objective.

[0268] Determine the difference between the classification information of the region of interest of the third target in the second target detection result and the classification information of the corresponding fourth region of interest in the third target detection result;

[0269] Determine the fourth target difference between the location information of the region of interest of the third target in the second target detection result and the location information of the corresponding fourth region of interest in the third target detection result;

[0270] Construct a tenth loss function based on the difference between the third objective and the fourth objective;

[0271] The second objective loss function is constructed based on the ninth and tenth loss functions.

[0272] The first update unit 306 is used to update the first network parameters of the student detection model based on the first objective loss function and the second objective loss function.

[0273] The iterative update unit 307 is used to update the second network parameters of the teacher detection model according to the updated first network parameters, and iteratively return to execute the input of the first image sample into the pre-trained detection model, output the corresponding first target detection result, so as to continuously update the second network parameters until the iteration condition is met and the trained teacher detection model is determined as the target detection model.

[0274] In some embodiments, the iterative update unit 307 is configured to:

[0275] The second network parameters of the teacher detection model are updated using the exponential moving average method and the adjusted first network parameters.

[0276] The iteration returns to the input of the first image sample into the detection model and outputs the corresponding first detection result, so as to continuously update the parameters of the third network until the third target loss function converges, thus obtaining the pre-trained detection model.

[0277] The data processing device may include an output unit and a generation unit, etc.

[0278] The output unit is used to input the image to be labeled into the target detection model trained by the training method of the target detection model described above, and output the corresponding fourth target detection result.

[0279] The generation unit is used to generate target label information corresponding to the image to be labeled based on the fourth target detection result.

[0280] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.

[0281] This application also provides a computer device, such as... Figure 6 As shown, it illustrates a schematic diagram of the server structure involved in an embodiment of this application. Specifically:

[0282] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 6 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0283] Processor 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 402, and by calling data stored in memory 402, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, processor 401 may include one or more processing cores; optionally, processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into processor 401.

[0284] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0285] The computer equipment also includes a power supply 403 that supplies power to the various components. Optionally, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0286] The computer device may also include an input unit 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0287] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402, thereby implementing the various method steps provided in the foregoing embodiments, as follows:

[0288] Acquire a first image sample and a second image sample. The first image sample carries label data, while the second image sample does not. Input the first image sample into a pre-trained detection model and output the corresponding first target detection result. Determine a first target loss function based on the first target detection result and the label data carried by the first image sample. Input the second image sample into a teacher detection model and a student detection model respectively, and output the corresponding second target detection result and third target detection result. Determine a second target loss function based on the difference between the second target detection result and the third target detection result. Update the first network parameters of the student detection model based on the first and second target loss functions. Update the second network parameters of the teacher detection model based on the updated first network parameters, and iterate back to continuously update the second network parameters by inputting the first image sample into the pre-trained detection model and outputting the corresponding first target detection result, until the iteration condition is met. The trained teacher detection model is then determined as the target detection model.

[0289] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the training method or data processing method for the object detection model above, which will not be repeated here.

[0290] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0291] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the object detection model training methods or data processing methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0292] Acquire a first image sample and a second image sample. The first image sample carries label data, while the second image sample does not. Input the first image sample into a pre-trained detection model and output the corresponding first target detection result. Determine a first target loss function based on the first target detection result and the label data carried by the first image sample. Input the second image sample into a teacher detection model and a student detection model respectively, and output the corresponding second target detection result and third target detection result. Determine a second target loss function based on the difference between the second target detection result and the third target detection result. Update the first network parameters of the student detection model based on the first and second target loss functions. Update the second network parameters of the teacher detection model based on the updated first network parameters, and iterate back to continuously update the second network parameters by inputting the first image sample into the pre-trained detection model and outputting the corresponding first target detection result, until the iteration condition is met. The trained teacher detection model is then determined as the target detection model.

[0293] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0294] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0295] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0296] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the data processing methods provided in the embodiments of this application, the beneficial effects that any of the data processing methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0297] The above provides a detailed description of the training method, apparatus, computer device, and storage medium for a target detection model provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A training method for an object detection model, characterized in that, include: Acquire a first image sample and a second image sample, wherein the first image sample carries label data and the second image sample does not carry label data; The first image sample is input into the pre-trained detection model, and the corresponding first target detection result is output. The first target loss function is determined based on the first target detection result and the label data carried by the first image sample; The second image sample is input into the teacher detection model and the student detection model respectively, and the corresponding second target detection result and third target detection result are output. The second target loss function is determined based on the difference between the second target detection result and the third target detection result; The first network parameters of the student detection model are updated based on the first objective loss function and the second objective loss function; The second network parameters of the teacher detection model are updated according to the updated first network parameters, and the process iteratively returns to the process of inputting the first image sample into the pre-trained detection model and outputting the corresponding first target detection result, so as to continuously update the second network parameters until the iteration condition is met and the trained teacher detection model is determined as the target detection model.

2. The training method according to claim 1, characterized in that, Before inputting the first image sample into the pre-trained detection model and outputting the corresponding first target detection result, the method further includes: The first image sample is input into the detection model, and the corresponding first detection result is output. A third target loss function is determined based on the difference between the first detection result and the label data; The third network parameters of the detection model are updated based on the third objective loss function, and the process iteratively returns to the previous step of inputting the first image sample into the detection model and outputting the corresponding first detection result. This process is repeated to continuously update the third network parameters until the third objective loss function converges, thus obtaining the pre-trained detection model. The first target network parameters of the pre-trained detection model are passed to the teacher detection model and the student detection model.

3. The training method according to claim 2, characterized in that, The step of determining the third target loss function based on the difference between the first detection result and the label data includes: A first loss function is determined based on the difference between the classification information of the first region in the first detection result and the classification information of the label region in the label data; A second loss function is determined based on the difference between the location information of the first region in the first detection result and the location information of the label region in the label data; A third loss function is determined based on the difference between the classification information of the first region of interest in the first detection result and the classification information of the region of interest of the labels in the label data; A fourth loss function is determined based on the difference between the location information of the first region of interest in the first detection result and the location information of the region of interest of the label in the label data; A third objective loss function is constructed based on the first, second, third, and fourth loss functions.

4. The training method according to claim 1, characterized in that, The step of determining the first target loss function based on the first target detection result and the label data carried by the first image sample includes: The fifth loss function is determined based on the difference between the classification information of the second region in the first target detection result and the classification information of the label region in the label data; A sixth loss function is determined based on the difference between the location information of the second region in the first target detection result and the location information of the label region in the label data; The seventh loss function is determined based on the difference between the classification information of the second region of interest in the first target detection result and the classification information of the label region of interest in the label data; The eighth loss function is determined based on the difference between the location information of the second region of interest in the first target detection result and the location information of the label region of interest in the label data; The first target loss function is constructed based on the fifth, sixth, seventh, and eighth loss functions.

5. The training method according to claim 1, characterized in that, The step of inputting the second image samples into the teacher detection model and the student detection model respectively, and outputting the corresponding second target detection result and third target detection result includes: The second image sample is weakly enhanced; The second image sample after weak enhancement is input into the teacher detection model, and the corresponding second target detection result is output. The second image sample is strongly enhanced; The enhanced second image sample is input into the student detection model, and the corresponding third object detection result is output.

6. The training method according to claim 5, characterized in that, The step of inputting the weakly enhanced second image sample into the teacher detection model and outputting the corresponding second target detection result includes: The second image sample after weak enhancement is input into the teacher detection model, and the corresponding classification information of the third region, the location information of the third region, the classification information of the third region of interest, and the location information of the third region of interest are output. The location information of the third region with a confidence level greater than the first preset threshold is determined as the location information of the third target region; Determine the classification information of the third target region corresponding to the location information of the third target region; The location information of the third region of interest with a confidence level greater than the second preset threshold is determined as the location information of the third target region of interest. Determine the classification information of the third target's region of interest corresponding to its location information; The classification information of the third target region, the location information of the third target region, the classification information of the third target region of interest, and the location information of the third target region of interest are determined as the second target detection result.

7. The training method according to claim 6, characterized in that, The step of inputting the enhanced second image sample into the student detection model and outputting the corresponding third target detection result includes: The enhanced second image sample is input into the student detection model, which outputs the corresponding classification information of the fourth region, the location information of the fourth region, the classification information of the fourth region of interest, and the location information of the fourth region of interest. The classification information of the fourth region, the location information of the fourth region, the classification information of the fourth region of interest, and the location information of the fourth region of interest are determined as the third target detection result.

8. The training method according to claim 7, characterized in that, The step of determining the second target loss function based on the difference between the second target detection result and the third target detection result includes: Determine the first target difference between the classification information of the third target region in the second target detection result and the classification information of the corresponding fourth region in the third target detection result; Determine the second target difference between the location information of the third target region in the second target detection result and the location information of the corresponding fourth region in the third target detection result; A ninth loss function is constructed based on the first target difference and the second target difference; Determine the difference between the classification information of the third target region of interest in the second target detection result and the classification information of the corresponding fourth target region of interest in the third target detection result; Determine the fourth target difference between the location information of the third target region of interest in the second target detection result and the location information of the corresponding fourth target region of interest in the third target detection result; A tenth loss function is constructed based on the differences between the third and fourth objectives. A second target loss function is constructed based on the ninth loss function and the tenth loss function.

9. The training method according to claim 1, characterized in that, The step of updating the second network parameters of the teacher detection model based on the updated first network parameters includes: The second network parameters of the teacher detection model are updated according to the exponential moving average method and the updated first network parameters.

10. A data processing method, characterized in that, include: The image to be labeled is input into the target detection model trained by the training method of the target detection model as described in claim 1, and the corresponding fourth target detection result is output. Based on the fourth target detection result, the target label information corresponding to the image to be labeled is generated.

11. A training device for an object detection model, characterized in that, include: The acquisition unit is used to acquire a first image sample and a second image sample, wherein the first image sample carries label data and the second image sample does not carry label data; The first output unit is used to input the first image sample into the pre-trained detection model and output the corresponding first target detection result; The first determining unit is configured to determine a first target loss function based on the first target detection result and the label data carried by the first image sample; The second output unit is used to input the second image sample into the teacher detection model and the student detection model respectively, and output the corresponding second target detection result and the third target detection result. The second determining unit is used to determine the second target loss function based on the difference between the second target detection result and the third target detection result; The first update unit is used to update the first network parameters of the student detection model based on the first target loss function and the second target loss function; The iterative update unit is used to update the second network parameters of the teacher detection model according to the updated first network parameters, and iteratively return to execute the input of the first image sample into the pre-trained detection model and output the corresponding first target detection result, so as to continuously update the second network parameters until the iteration condition is met and the trained teacher detection model is determined as the target detection model.

12. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the training method of the target detection model according to any one of claims 1 to 9 or the data processing method according to claim 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform steps in the training method of the target detection model according to any one of claims 1 to 9 or the data processing method according to claim 10.

Images