Target detection method and device

Abstract

The present application provides a target detection method and device, including: adding pseudo labels to unlabeled data, dividing the pseudo labels into high-quality pseudo labels and uncertain pseudo labels; inputting the unlabeled data into an initial learning model to obtain a first prediction value; determining a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label, and determining a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label; inputting the unlabeled data into an initial management model to obtain a second prediction value, and determining a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label; training the initial management model based on the first prediction label, the first prediction box, the second prediction label, the second prediction box, the third prediction label, and the third prediction box to obtain a target management model, which is used to perform target detection on the data to be detected. Through the solution of the present application, the acquisition of a large amount of labeled data is avoided.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a target detection method and device. Background Art

[0002] Machine learning is a path to artificial intelligence and a multidisciplinary field, encompassing probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning focuses on algorithm design, enabling computers to automatically learn patterns from data and use these patterns to make predictions about unknown data. Machine learning has found a wide range of applications, including deep learning, data mining, computer vision, natural language processing, biometrics, search engines, medical diagnostics, speech recognition, and handwriting recognition.

[0003] In order to use machine learning to implement artificial intelligence processing, a training data set can be constructed. The training data set includes a large amount of labeled data (such as image data, that is, images with calibrated frames and calibrated categories). A machine learning model is trained based on the training data set, such as a machine learning model with target detection function. The machine learning model can be used to perform target detection on the data to be detected. For example, the target frame in the data to be detected can be detected, and the target category can be identified, such as vehicle category, animal category, electronic product category, etc.

[0004] To improve the performance of machine learning models, a large amount of labeled data is required. The more labeled data there is, the better the performance of the trained machine learning model. However, obtaining labeled data requires data annotation, which consumes a lot of human resources. Summary of the Invention

[0005] The present application provides a target detection method, the method comprising:

[0006] Obtaining an initial management model and an initial learning model, adding pseudo labels to unlabeled data based on the initial management model, and dividing the pseudo labels into high-quality pseudo labels and uncertain pseudo labels;

[0007] Inputting unlabeled data into the initial learning model to obtain a first prediction value corresponding to the unlabeled data; determining a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label, and determining a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label;

[0008] Inputting the unlabeled data into the initial management model to obtain a second prediction value corresponding to the unlabeled data, and determining a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label;

[0009] Training an initial management model based on the first prediction label, the first prediction frame, the second prediction label, the second prediction frame, the third prediction label, and the third prediction frame to obtain a trained target management model;

[0010] The target management model is used to perform target detection on the data to be detected.

[0011] The present application provides a target detection device, comprising:

[0012] an acquisition module, configured to acquire an initial management model and an initial learning model, add pseudo labels to unlabeled data based on the initial management model, and classify the pseudo labels into high-quality pseudo labels and uncertain pseudo labels;

[0013] A determination module is configured to input the unlabeled data into an initial learning model to obtain a first prediction value corresponding to the unlabeled data; determine a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label, and determine a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label; input the unlabeled data into an initial management model to obtain a second prediction value corresponding to the unlabeled data, and determine a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label;

[0014] A processing module is used to train the initial management model based on the first prediction label, the first prediction box, the second prediction label, the second prediction box, the third prediction label and the third prediction box to obtain a trained target management model; wherein, the target management model is used to perform target detection on the data to be detected.

[0015] The present application provides a target detection device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0016] The processor is used to execute machine-executable instructions to implement the target detection method of the above example.

[0017] It can be seen from the above technical solutions that in the embodiment of the present application, the target management model can be obtained based on the training of unlabeled data, that is, the target management model can also be obtained by training with a small amount of labeled data and a large amount of unlabeled data, thereby avoiding the acquisition of a large amount of labeled data, reducing the workload of the annotation operation, saving human resources, and the target management model has relatively good performance and high reliability. It can improve the robustness to noise samples in the training process on the basis of effective use of pseudo-labels, and the target management model has a very large improvement compared with the baseline model. Setting pseudo-labels of different proportions as high-quality pseudo-labels can achieve relatively good training results, has good robustness to noise samples, and is insensitive to hyperparameters. A semi-supervised target detection training method is proposed, which draws on the idea of ​​maintaining consistency of features of the same image after different augmentations in unsupervised representation learning, establishes consistency constraints for the regression boxes and classifications under different augmentations of unlabeled data, combines pseudo-labels and consistency constraints, and uses pseudo-labels as true categories for reliable target boxes, while establishing consistency comparison losses for different prediction results (or features) for uncertain targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present application or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present application.

[0019] Figure 1 is a flow chart of a target detection method in one embodiment of the present application;

[0020] Figure 2 is a flow chart of a target detection method in one embodiment of the present application;

[0021] Figure 3 This is a schematic structural diagram of a target detection device in one embodiment of the present application;

[0022] Figure 4 This is a hardware structure diagram of a target detection device in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The terms used in the embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms "a," "the," and "the" used in this application and claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items.

[0024] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" used may also be interpreted as "at the time of" or "when" or "in response to determining".

[0025] In the embodiment of the present application, a target detection method is proposed, which can be applied to a target detection device. The target detection device can be any type of device, such as a server, a terminal device, a management device, etc., without limitation. Figure 1 FIG. 5 is a flow chart of a target detection method. The target detection method of this embodiment may be a semi-supervised target detection method, which may include:

[0026] Step 101: Obtain an initial management model and an initial learning model, add pseudo labels to unlabeled data based on the initial management model, and divide the pseudo labels into high-quality pseudo labels and uncertain pseudo labels.

[0027] Exemplarily, obtaining the initial management model and the initial learning model may include, but is not limited to: obtaining a baseline model through training with labeled data, and generating the initial management model and the initial learning model based on the baseline model. The network structure of the initial management model may be the same as that of the baseline model, and the network parameters of the initial management model may be the same as or different from those of the baseline model. The network structure of the initial learning model may be the same as that of the baseline model, and the network parameters of the initial learning model may be the same as or different from those of the baseline model. The network structure of the initial management model may be the same as that of the initial learning model, and the network parameters of the initial management model may be the same as or different from those of the initial learning model.

[0028] Exemplarily, adding pseudo-labels to unlabeled data based on the initial management model and classifying the pseudo-labels into high-quality pseudo-labels and uncertain pseudo-labels may include, but is not limited to: for each unlabeled data item, the unlabeled data item may be input into the initial management model to obtain the pseudo-label corresponding to the unlabeled data item and the probability value corresponding to the pseudo-label. On this basis, for each category supported by the initial management model, all pseudo-labels corresponding to the category are sorted based on the probability values ​​corresponding to all pseudo-labels corresponding to the category. Based on the sorting results, K pseudo-labels with large probability values ​​are selected as high-quality pseudo-labels, and the remaining pseudo-labels excluding the high-quality pseudo-labels are regarded as uncertain pseudo-labels. K is a positive integer.

[0029] Step 102: Input the unlabeled data into the initial learning model to obtain a first prediction value corresponding to the unlabeled data; determine a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label; and determine a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label. Input the unlabeled data into the initial management model to obtain a second prediction value corresponding to the unlabeled data; and determine a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label.

[0030] Exemplarily, a first data augmentation may be performed on the unlabeled data, and the unlabeled data after the first data augmentation is input into the initial learning model to obtain a first prediction value corresponding to the unlabeled data.

[0031] Exemplarily, a second data augmentation may be performed on the unlabeled data, and the unlabeled data after the second data augmentation is input into the initial management model to obtain a second prediction value corresponding to the unlabeled data.

[0032] Illustratively, the first data augmentation method and the second data augmentation method may be different.

[0033] Step 103: Train the initial management model based on the first predicted label, the first predicted frame, the second predicted label, the second predicted frame, the third predicted label, and the third predicted frame to obtain a trained target management model; wherein the target management model is used to perform target detection on the data to be detected. For example, the following steps can be used to train the initial management model to obtain the target management model:

[0034] Step 1031: Determine a first loss value based on the first predicted label and the first predicted box, and determine a second loss value based on the second predicted label, the second predicted box, the third predicted label, and the third predicted box.

[0035] Exemplarily, determining the second loss value based on the second prediction label, the second prediction box, the third prediction label, and the third prediction box may include, but is not limited to: if the second prediction label includes C first probability values ​​corresponding to the C categories supported by the initial learning model, and the third prediction label includes C second probability values ​​corresponding to the C categories supported by the initial management model, then determining the category loss value of the consistency constraint based on the C first probability values ​​and the C second probability values, where C can be a positive integer greater than 1. In addition, determining a first probability distribution of the coordinate point offset corresponding to the second prediction box, determining a second probability distribution of the coordinate point offset corresponding to the third prediction box, and determining the coordinate box loss value of the consistency constraint based on the first probability distribution and the second probability distribution. The second loss value is determined based on the category loss value and the coordinate box loss value.

[0036] In one possible implementation, determining the coordinate frame loss value of the consistency constraint based on the first probability distribution and the second probability distribution may include, but is not limited to: determining the relative entropy between the first probability distribution and the second probability distribution, and determining the coordinate frame loss value of the consistency constraint based on the relative entropy.

[0037] Step 1032: Train the initial management model based on the first loss value and the second loss value to obtain a trained target management model; wherein the target management model is used to perform target detection on the data to be detected.

[0038] Exemplarily, before step 1032, the labeled data can also be input into the initial learning model to obtain a third prediction value corresponding to the labeled data, and a fourth prediction label and a fourth prediction box are determined based on the third prediction value, and a third loss value is determined based on the fourth prediction label and the fourth prediction box.

[0039] On this basis, for step 1032, the initial management model can be trained based on the first loss value, the second loss value and the third loss value to obtain a trained target management model.

[0040] Exemplarily, the initial management model is trained based on the first loss value and the second loss value to obtain a trained target management model, which may include but is not limited to: adjusting the network parameters of the initial learning model based on the first loss value and the second loss value to obtain an adjusted learning model; adjusting the network parameters of the initial management model based on the network parameters of the adjusted learning model to obtain an adjusted management model; if the adjusted management model has not converged, determining the adjusted learning model as the initial learning model, determining the adjusted management model as the initial management model, and returning to perform the operation of adding pseudo labels to unlabeled data based on the initial management model, and dividing the pseudo labels into high-quality pseudo labels and uncertain pseudo labels (i.e., step 101); if the adjusted management model has converged, determining the adjusted management model as the target management model.

[0041] Exemplarily, adjusting the network parameters of the initial management model based on the network parameters of the adjusted learning model to obtain the adjusted management model may include but is not limited to: determining parameter correction values ​​of the network parameters based on the network parameters of the adjusted learning model and the configured proportional coefficients, and adjusting the network parameters of the initial management model based on the parameter correction values ​​to obtain the adjusted management model.

[0042] It can be seen from the above technical solutions that in the embodiment of the present application, the target management model can be obtained based on the training of unlabeled data, that is, the target management model can also be obtained by training with a small amount of labeled data and a large amount of unlabeled data, thereby avoiding the acquisition of a large amount of labeled data, reducing the workload of the annotation operation, saving human resources, and the target management model has relatively good performance and high reliability. It can improve the robustness to noise samples in the training process on the basis of effective use of pseudo-labels, and the target management model has a very large improvement compared with the baseline model. Setting pseudo-labels of different proportions as high-quality pseudo-labels can achieve relatively good training results, has good robustness to noise samples, and is insensitive to hyperparameters. A semi-supervised target detection training method is proposed, which draws on the idea of ​​maintaining consistency of features of the same image after different augmentations in unsupervised representation learning, establishes consistency constraints for the regression boxes and classifications under different augmentations of unlabeled data, combines pseudo-labels and consistency constraints, and uses pseudo-labels as true categories for reliable target boxes, while establishing consistency comparison losses for different prediction results (or features) for uncertain targets.

[0043] The following describes the technical solutions of the embodiments of the present application in conjunction with specific application scenarios.

[0044] In the embodiment of the present application, a target detection method is proposed, which is a semi-supervised target detection method based on consistency constraints. In the semi-supervised target detection method, partial labeled data can be combined with large-scale unlabeled data for training to achieve performance close to that of full labeled data. Figure 2 FIG. 1 is a flow chart of a target detection method, which may include:

[0045] Step 201: Use labeled data to train and obtain a baseline model.

[0046] Exemplarily, a training data set can be pre-constructed, which can include multiple labeled data (such as labeled images). For each labeled data, the labeled data corresponds to calibration information, which includes but is not limited to a calibration box (such as when the calibration box is a rectangular calibration box, it can be the coordinates of the four vertices of the rectangular calibration box) and a calibration category (that is, the category of the target object in the calibration box).

[0047] For example, an initial network model can be pre-acquired. This initial network model can be a machine learning model, such as a machine learning model based on a deep learning algorithm or a machine learning model based on a neural network. The type of machine learning model is not limited. This embodiment does not limit the structure of the initial network model. Regarding the function of the initial network model, the initial network model can implement target detection.

[0048] For example, the initial network model can be trained based on the labeled data in the training data set. There is no restriction on the training process. The trained initial network model is called a baseline model. That is to say, a baseline model can be obtained by training using the labeled data in the training data set.

[0049] During the training process of the initial network model, the labeled data is input into the initial network model, and the initial network model processes the labeled data to obtain the prediction box and prediction label corresponding to the labeled data. The prediction box is used to indicate the location of the target object, such as the coordinates of the four vertices, and the prediction label is used to indicate the category of the target object. For example, if the initial network model supports category 1 and category 2, then when the prediction label is 0, it indicates that the category of the target object is category 1, and when the prediction label is 1, it indicates that the category of the target object is category 2.

[0050] The category loss value can be determined based on the predicted label corresponding to the labeled data and the calibrated category (i.e., the true category) corresponding to the labeled data, and the coordinate frame loss value can be determined based on the predicted box corresponding to the labeled data and the calibrated box (i.e., the true box) corresponding to the labeled data. Then, the target loss value of the initial network model is determined based on the category loss value and the coordinate frame loss value. For example, the target loss value of the initial network model can be determined using the following formula: L = L loc +L cls In the above formula, L represents the target loss value of the initial network model, L loc Represents the coordinate frame loss value, L cls Represents the class loss value.

[0051] Among them, for the category loss value L cls If the predicted label matches the calibration category, if the two are the same, the category loss value is small, such as the category loss value is the minimum loss value; if the predicted label does not match the calibration category, if the two are different, the category loss value is large, such as the category loss value is the maximum loss value. Of course, the above is just an example of determining the category loss value, and there is no restriction on this.

[0052] Among them, for the coordinate frame loss value L loc If the predicted box matches the calibration box, such as the coordinates of the four vertices of the predicted box are the same as the coordinates of the four vertices of the calibration box, the coordinate box loss value is smaller, and if the coordinate box loss value is the minimum loss value, if the predicted box does not match the calibration box, such as the coordinates of the four vertices of the predicted box are different from the coordinates of the four vertices of the calibration box, then the degree of closeness between the coordinates of the four vertices of the predicted box and the coordinates of the four vertices of the calibration box is determined. If the degree of closeness indicates that the closer the predicted box is to the calibration box, the smaller the coordinate box loss value is, if the degree of closeness indicates that the difference between the predicted box and the calibration box is greater, the coordinate box loss value is greater.

[0053] For example, determine the probability distribution (such as Gaussian distribution) of the coordinate point offsets corresponding to the four vertex coordinates of the prediction frame, that is, each vertex coordinate of the prediction frame corresponds to a probability distribution, such as the upper left vertex coordinate corresponds to the probability distribution a1, the upper right vertex coordinate corresponds to the probability distribution a2, the lower right vertex coordinate corresponds to the probability distribution a3, and the lower left vertex coordinate corresponds to the probability distribution a4. And, determine the probability distribution of the coordinate point offsets corresponding to the four vertex coordinates of the calibration frame, that is, each vertex coordinate of the calibration frame corresponds to a probability distribution, such as the upper left vertex coordinate corresponds to the probability distribution b1, the upper right vertex coordinate corresponds to the probability distribution b2, the lower right vertex coordinate corresponds to the probability distribution b3, and the lower left vertex coordinate corresponds to the probability distribution b4. When determining the probability distribution corresponding to the vertex coordinates, the probability distribution can be represented by the mean and variance. For example, the probability distribution corresponding to the vertex coordinate x (such as the vertex coordinates of the prediction frame or the vertex coordinates of the calibration frame) is expressed as N(μ tx ,∑ tx ).

[0054] Based on the probability distribution corresponding to the coordinates of the four vertices of the prediction frame and the probability distribution corresponding to the coordinates of the four vertices of the calibration frame, the coordinate frame loss value between the prediction frame and the calibration frame can be calculated. For example, the negative log-likelihood loss value is calculated based on the probability distribution a1 and the probability distribution b1, the negative log-likelihood loss value is calculated based on the probability distribution a2 and the probability distribution b2, the negative log-likelihood loss value is calculated based on the probability distribution a3 and the probability distribution b3, and the negative log-likelihood loss value is calculated based on the probability distribution a4 and the probability distribution b4. The coordinate frame loss value is determined based on the above four negative log-likelihood loss values, such as the mean of the four negative log-likelihood loss values ​​as the coordinate frame loss value, or the sum of the four negative log-likelihood loss values ​​as the coordinate frame loss value. Of course, the above is just an example and there is no limitation to this, as long as the coordinate frame loss value can be obtained based on the probability distribution.

[0055] During the training process of the initial network model, after obtaining the target loss value of the initial network model, the network parameters of the initial network model can be adjusted based on the target loss value, such as by using the gradient descent method or the like. This embodiment does not limit this adjustment process, and uses the adjusted network model as the initial network model, and returns to execute the operation of inputting labeled data into the initial network model, and so on, until the initial network model has converged (such as the number of iterations of the initial network model reaches the number threshold, or the target loss value is less than the loss value threshold), and uses the converged initial network model as the baseline model. At this point, a baseline model is obtained by training with labeled data.

[0056] Step 202: Generate an initial management model and an initial learning model based on the baseline model.

[0057] For example, an initial management model (also referred to as an initial teacher model) can be generated based on the baseline model. The network structure of the initial management model can be the same as that of the baseline model, and the network parameters of the initial management model can be the same as or different from those of the baseline model. For example, the baseline model can be directly used as the initial management model, or the network parameters of the baseline model can be adjusted, and the baseline model with the adjusted network parameters can be used as the initial management model.

[0058] For example, an initial learning model (also referred to as an initial student model) can be generated based on the baseline model. The network structure of the initial learning model can be the same as that of the baseline model, and the network parameters of the initial learning model can be the same as or different from those of the baseline model. For example, the baseline model can be directly used as the initial learning model, or the network parameters of the baseline model can be adjusted, and the baseline model with the adjusted network parameters can be used as the initial learning model.

[0059] Step 203: Add pseudo labels to the unlabeled data based on the initial management model, and classify the pseudo labels into high-quality pseudo labels and uncertain pseudo labels. For example, the pseudo labels corresponding to the unlabeled data are determined based on the initial management model, and the pseudo labels are classified as high-quality pseudo labels or uncertain pseudo labels.

[0060] For example, a sample dataset can be pre-built, which can include multiple unlabeled data (such as unlabeled images), that is, the unlabeled data is added to the sample dataset. For each unlabeled data, the unlabeled data has no calibration information, that is, no corresponding calibration box and calibration category.

[0061] Exemplarily, for each unlabeled data in the sample data set, the unlabeled data can be input into the initial management model, which processes the unlabeled data to obtain the prediction box corresponding to the unlabeled data, the prediction label corresponding to the prediction box corresponding to the unlabeled data, and the probability value corresponding to the prediction label (that is, the probability value that the target object in the prediction box is the prediction label).

[0062] For example, assuming that the initial management model supports target detection of category 1, category 2, and category 3, after processing the unlabeled data, the initial management model can obtain a prediction box and a probability vector corresponding to the prediction box. For example, the probability vector can be [0.9, 0.06, 0.04]. Based on the probability vector, it can be known that the prediction label corresponding to the prediction box is category 1, and the probability value corresponding to the prediction label is 0.9.

[0063] For each unlabeled data item, the unlabeled data item may correspond to P prediction boxes, where P is a positive integer. Each prediction box corresponds to a prediction label, that is, P prediction boxes correspond to P prediction labels, and each prediction label corresponds to a probability value. In addition, each prediction box may also correspond to a probability value. For example, after the initial management model processes the unlabeled data, it obtains prediction box 1 and the probability value of prediction box 1, prediction label 1 corresponding to prediction box 1 and the probability value of prediction label 1, as well as prediction box 2 and the probability value of prediction box 2, prediction label 2 corresponding to prediction box 2 and the probability value of prediction label 2.

[0064] Among them, the prediction box corresponding to the unlabeled data and the prediction label corresponding to the prediction box can be called pseudo-labels, and the probability value corresponding to the prediction label and the probability value corresponding to the prediction box can be called probability values ​​corresponding to the pseudo-label. Therefore, after the unlabeled data is input into the initial management model, the pseudo-label corresponding to the unlabeled data (i.e., the prediction box and the prediction label) and the probability value corresponding to the pseudo-label can be obtained.

[0065] In summary, after inputting multiple unlabeled data in the sample data set into the initial management model, a large number of pseudo labels can be obtained. For each pseudo label, the probability value corresponding to the pseudo label can be obtained.

[0066] Exemplarily, for each category supported by the initial management model, all pseudo-labels corresponding to the category are sorted based on the probability values ​​corresponding to all pseudo-labels corresponding to the category (the category corresponding to the pseudo-label is obtained based on the predicted label of the pseudo-label); based on the sorting results, K pseudo-labels with large probability values ​​are selected as high-quality pseudo-labels, and the remaining pseudo-labels except the high-quality pseudo-labels are regarded as uncertain pseudo-labels.

[0067] Assuming that the initial management model supports category 1, category 2, and category 3, for each pseudo-label, if the predicted label in the pseudo-label is category 1, then the pseudo-label corresponds to category 1; if the predicted label in the pseudo-label is category 2, then the pseudo-label corresponds to category 2; if the predicted label in the pseudo-label is category 3, then the pseudo-label corresponds to category 3. In summary, for each category supported by the initial management model, all pseudo-labels corresponding to the category can be obtained (i.e., predicted boxes and predicted labels), such as category 1 corresponds to pseudo-labels c1-c100, category 2 corresponds to pseudo-labels c101-c300, and category 3 corresponds to pseudo-labels c301-c600.

[0068] Based on the probability values ​​corresponding to all pseudo-labels corresponding to the category, all pseudo-labels corresponding to the category are sorted. For example, based on the probability values ​​corresponding to the predicted labels in the pseudo-labels, all pseudo-labels corresponding to the category are sorted in descending order according to the probability values ​​corresponding to the predicted labels, or, in descending order according to the probability values ​​corresponding to the predicted labels, all pseudo-labels corresponding to the category are sorted. For another example, the probability product value (or probability average) between the probability value corresponding to the predicted label in the pseudo-label and the probability value corresponding to the prediction box is calculated, and all pseudo-labels corresponding to the category are sorted in descending order according to the probability product value, or, in descending order according to the probability product value, all pseudo-labels corresponding to the category are sorted. Of course, other sorting methods can also be used, and there is no limitation on this.

[0069] Based on the sorting results, K pseudo-labels with large probability values ​​can be selected as high-quality pseudo-labels, and the remaining pseudo-labels except the high-quality pseudo-labels can be used as uncertain pseudo-labels. For example, if the probability values ​​(or probability product values) corresponding to the predicted labels are sorted from large to small, the K pseudo-labels with the highest order can be selected as high-quality pseudo-labels, and the remaining pseudo-labels can be used as uncertain pseudo-labels. If the probability values ​​(or probability product values) corresponding to the predicted labels are sorted from small to large, the K pseudo-labels with the lowest order can be selected as high-quality pseudo-labels, and the remaining pseudo-labels can be used as uncertain pseudo-labels.

[0070] For the pseudo labels c1-c100 corresponding to category 1, sort the pseudo labels c1-c100 according to the probability values ​​corresponding to the predicted labels (the probability values ​​corresponding to category 1) from large to small, select the top K pseudo labels (such as c1-c10) as high-quality pseudo labels, and use the remaining pseudo labels (such as c11-c100) as uncertain pseudo labels. For the pseudo labels c101-c300 corresponding to category 2, sort the pseudo labels c101-c300 according to the probability values ​​corresponding to the predicted labels (the probability values ​​corresponding to category 2) from large to small, select the top K pseudo labels as high-quality pseudo labels, and use the remaining pseudo labels as uncertain pseudo labels. This process can be repeated to obtain high-quality pseudo labels and uncertain pseudo labels for each category.

[0071] The value of K can be configured based on experience or determined based on the total number of pseudo-labels corresponding to the category, such as total number * M, where M is a value between 0 and 1. It can be configured based on experience, such as 20%, 30%, etc. On this basis, since category 1 corresponds to 100 pseudo-labels, the value of K is 20, that is, the top 20 pseudo-labels are selected from all pseudo-labels corresponding to category 1 as high-quality pseudo-labels, and the remaining 80 pseudo-labels are used as uncertain pseudo-labels. Since category 2 corresponds to 200 pseudo-labels, the value of K is 40, that is, the top 40 pseudo-labels are selected from all pseudo-labels corresponding to category 2 as high-quality pseudo-labels, and the remaining 160 pseudo-labels are used as uncertain pseudo-labels, and so on.

[0072] In summary, all pseudo-labels corresponding to all unlabeled data in the sample dataset can be divided into high-quality pseudo-labels and uncertain pseudo-labels. High-quality pseudo-labels can be used as reliable labels. In subsequent training, high-quality pseudo-labels can be treated as labeled data. That is, high-quality pseudo-labels correspond to calibrated boxes and calibrated categories. For example, the predicted box output by the initial management model is used as the calibrated box of the high-quality pseudo-label, and the predicted label output by the initial management model is used as the calibrated category of the high-quality pseudo-label.

[0073] In summary, we can obtain labeled data, high-quality pseudo-labels, and uncertain pseudo-labels. Based on this, we can perform joint training on labeled data and unlabeled data (this unlabeled data can be unlabeled data with high-quality pseudo-labels and unlabeled data with uncertain pseudo-labels), that is, semi-supervised joint training. During the joint training process, the ratio of labeled data to unlabeled data can be m:n, that is, the ratio of the total amount of labeled data to the total amount of unlabeled data is m:n. This m:n ratio can be configured based on experience, and there is no restriction on the value of this m:n ratio, such as 1:1, 1:2, 2:1, etc.

[0074] For the semi-supervised training process, continue to refer to Figure 2 As shown, the following steps may be included:

[0075] Step 204: perform data augmentation on the labeled data, input the augmented labeled data into the initial learning model, obtain a third prediction value corresponding to the labeled data, determine a fourth prediction label and a fourth prediction box based on the third prediction value, and determine a third loss value based on the fourth prediction label and the fourth prediction box.

[0076] For each labeled data, data augmentation can be performed on the labeled data by using methods such as spatial transformation and / or color transformation. There is no restriction on this process, and the labeled data after data augmentation is obtained.

[0077] During the joint training process, the augmented labeled data can be input into the initial learning model, which processes the labeled data to obtain a third prediction value corresponding to the labeled data. The third prediction value can include a prediction box and a prediction label. For convenience, the prediction box is referred to as the fourth prediction box, and the prediction label is referred to as the fourth prediction label. That is, the fourth prediction label and the fourth prediction box can be determined based on the third prediction value. The fourth prediction box is used to represent the location of the target object, such as the coordinates of four vertices, and the fourth prediction label is used to represent the category of the target object, such as category 1, category 2, or category 3.

[0078] For labeled data, the labeled data corresponds to calibration information, such as a calibration box and a calibration category. On this basis, the category loss value can be determined based on the fourth predicted label corresponding to the labeled data and the calibration category corresponding to the labeled data, and the coordinate frame loss value can be determined based on the fourth predicted box corresponding to the labeled data and the calibration box corresponding to the labeled data. Then, the third loss value is determined based on the category loss value and the coordinate frame loss value. For example, the third loss value is determined using the following formula: L = L loc +L cls , L is used to represent the third loss value, L loc Used to represent the coordinate frame loss value, L cls Used to represent the class loss value.

[0079] Among them, for the category loss value L cls If the fourth predicted label matches the calibration category, the category loss value is small, such as the category loss value is the minimum loss value; if the fourth predicted label does not match the calibration category, the category loss value is large, such as the category loss value is the maximum loss value.

[0080] Among them, for the coordinate frame loss value L loc If the fourth prediction frame matches the calibration frame, the coordinate frame loss value is smaller, such as the coordinate frame loss value is the minimum loss value. If the fourth prediction frame does not match the calibration frame, the degree of closeness between the four vertex coordinates of the fourth prediction frame and the four vertex coordinates of the calibration frame can be determined. If the degree of closeness indicates that the fourth prediction frame is closer to the calibration frame, the smaller the coordinate frame loss value is. If the degree of closeness indicates that the fourth prediction frame is more different from the calibration frame, the greater the coordinate frame loss value is.

[0081] For example, determine the probability distribution of the coordinate point offsets corresponding to the four vertex coordinates of the fourth prediction frame, that is, each vertex coordinate of the fourth prediction frame corresponds to a probability distribution, and determine the probability distribution of the coordinate point offsets corresponding to the four vertex coordinates of the calibration frame, that is, each vertex coordinate of the calibration frame corresponds to a probability distribution. When determining the probability distribution corresponding to the vertex coordinates, the probability distribution is represented by the mean and variance. For example, the probability distribution corresponding to the vertex coordinate x (such as the vertex coordinates of the prediction frame or the vertex coordinates of the calibration frame) is N(μ tx ,∑ tx ).

[0082] Based on the probability distribution corresponding to the coordinates of the four vertices of the fourth prediction frame and the probability distribution corresponding to the coordinates of the four vertices of the calibration frame, the coordinate frame loss value between the fourth prediction frame and the calibration frame can be calculated. For example, the negative log-likelihood loss value can be calculated based on the probability distribution corresponding to the fourth prediction frame and the probability distribution corresponding to the calibration frame, and the coordinate frame loss value can be determined based on the negative log-likelihood loss value.

[0083] To sum up, the category loss value and the coordinate frame loss value corresponding to the labeled data can be obtained, and the third loss value corresponding to the labeled data can be determined based on the category loss value and the coordinate frame loss value.

[0084] Step 205: Perform a first data augmentation on the unlabeled data, input the unlabeled data after the first data augmentation into the initial learning model to obtain a first prediction value corresponding to the unlabeled data; determine a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label, and determine a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label.

[0085] Exemplarily, for each unlabeled data, a first data augmentation may be performed on the unlabeled data by using a method such as space transformation and / or color transformation to obtain unlabeled data after the first data augmentation.

[0086] Exemplarily, during the joint training process, the unlabeled data after the first data augmentation can be input into the initial learning model, and the initial learning model processes the unlabeled data to obtain a first prediction value corresponding to the unlabeled data. The first prediction value may include a prediction box and a prediction label.

[0087] Referring to step 203, pseudo labels have been added to the unlabeled data, and the pseudo labels are divided into high-quality pseudo labels and uncertain pseudo labels. The high-quality pseudo labels can correspond to the predicted box (i.e., as the calibration box) and the predicted label (i.e., as the calibration category), and the uncertain pseudo labels can correspond to the predicted box and the predicted label.

[0088] On this basis, if the prediction box in the first prediction value matches the prediction box corresponding to the high-quality pseudo label, that is, the two represent the prediction boxes of the same area in the same unlabeled data (such as an unlabeled image), then this first prediction value is the first prediction value corresponding to the high-quality pseudo label, and the prediction label in the first prediction value is used as the first prediction label, and the prediction box in the first prediction value is used as the first prediction box, that is, the first prediction label and the first prediction box are determined based on the first prediction value corresponding to the high-quality pseudo label.

[0089] If the prediction box in the first prediction value matches the prediction box corresponding to the uncertain pseudo-label, that is, the two represent the prediction boxes of the same area in the same unlabeled data (such as an unlabeled image), then this first prediction value is the first prediction value corresponding to the uncertain pseudo-label, and the prediction label in the first prediction value is used as the second prediction label, and the prediction box in the first prediction value is used as the second prediction box, that is, the second prediction label and the second prediction box are determined based on the first prediction value corresponding to the uncertain pseudo-label.

[0090] Among them, the second prediction box is used to indicate the location of the target object, such as 4 vertex coordinates, and the second prediction label is used to indicate the category of the target object. For the second prediction label, it can include C first probability values ​​corresponding to the C categories supported by the initial learning model, and C can be a positive integer greater than 1. For example, assuming that the initial learning model supports three categories, namely category 1, category 2, and category 3, the second prediction label can include the first probability value corresponding to category 1 (such as 0.5), the first probability value corresponding to category 2 (such as 0.3), and the first probability value corresponding to category 3 (such as 0.2), that is, the second prediction label is [0.5, 0.3, 0.2].

[0091] Step 206: perform a second data augmentation on the unlabeled data, input the unlabeled data after the second data augmentation into the initial management model, and obtain a second prediction value corresponding to the unlabeled data; determine a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label.

[0092] For example, for each unlabeled data, a second data augmentation may be performed on the unlabeled data using a spatial transformation and / or color transformation method to obtain the second data-augmented unlabeled data. The first data augmentation method and the second data augmentation method may be different, i.e., two data augmented unlabeled data are obtained by performing data augmentation twice on the same unlabeled data using different augmentation methods. One unlabeled data is input into the initial learning model, and the other unlabeled data is input into the initial management model.

[0093] Exemplarily, during the joint training process, the unlabeled data after the second data augmentation can be input into the initial management model, and the initial management model processes the unlabeled data to obtain a second prediction value corresponding to the unlabeled data, which may include a prediction box and a prediction label.

[0094] Referring to step 203, pseudo labels have been added to the unlabeled data, and the pseudo labels are divided into high-quality pseudo labels and uncertain pseudo labels. The high-quality pseudo labels can correspond to prediction boxes and prediction labels, and the uncertain pseudo labels can correspond to prediction boxes and prediction labels. On this basis, if the prediction box in the second prediction value matches the prediction box corresponding to the high-quality pseudo label, the prediction box and prediction label in the second prediction value are no longer considered, that is, they do not participate in subsequent training. If the prediction box in the second prediction value matches the prediction box corresponding to the uncertain pseudo label, that is, the two represent the prediction box of the same area in the same unlabeled data (such as an unlabeled image), then this second prediction value is the second prediction value corresponding to the uncertain pseudo label, the prediction label in the second prediction value is used as the third prediction label, and the prediction box in the second prediction value is used as the third prediction box, that is, the third prediction label and the third prediction box are determined based on the second prediction value corresponding to the uncertain pseudo label.

[0095] Among them, the third prediction box is used to indicate the location of the target object, such as 4 vertex coordinates, and the third prediction label is used to indicate the category of the target object. For the third prediction label, it can include C second probability values ​​corresponding to the C categories supported by the initial management model, and C can be a positive integer greater than 1. For example, assuming that the initial management model supports three categories, namely category 1, category 2, and category 3, the third prediction label can include the second probability value corresponding to category 1 (such as 0.6), the second probability value corresponding to category 2 (such as 0.2), and the second probability value corresponding to category 3 (such as 0.2), that is, the third prediction label is [0.6, 0.2, 0.2].

[0096] Step 207: Determine a first loss value based on the first predicted label and the first predicted box.

[0097] Since the first predicted label and the first predicted box are the predicted label and the predicted box corresponding to the high-quality pseudo label, and the high-quality pseudo label also corresponds to calibration information, such as the calibration box and the calibration category (see step 203, the predicted box corresponding to the high-quality pseudo label is used as the calibration box, and the predicted label corresponding to the high-quality pseudo label is used as the calibration category), on this basis, the category loss value can be determined based on the first predicted label corresponding to the high-quality pseudo label and the calibration category corresponding to the high-quality pseudo label, and the coordinate box loss value can be determined based on the first predicted box corresponding to the high-quality pseudo label and the calibration box corresponding to the high-quality pseudo label. Then, the first loss value can be determined based on the category loss value and the coordinate box loss value. Among them, the determination process of the first loss value can refer to the determination process of the third loss value mentioned above, and will not be repeated here.

[0098] Step 208: Determine a second loss value based on the second predicted label, the second predicted box, the third predicted label, and the third predicted box. Exemplarily, since the second predicted label and the second predicted box are the predicted label and the predicted box corresponding to the uncertain pseudo label (output by the initial learning model), and the third predicted label and the third predicted box are also the predicted label and the predicted box corresponding to the uncertain pseudo label (output by the initial management model), on this basis, the category loss value can be determined based on the second predicted label corresponding to the uncertain pseudo label and the third predicted label corresponding to the uncertain pseudo label, and the coordinate box loss value can be determined based on the second predicted box corresponding to the uncertain pseudo label and the third predicted box corresponding to the uncertain pseudo label. Then, the second loss value can be determined based on the category loss value and the coordinate box loss value.

[0099] Exemplarily, the second prediction label includes C first probability values, and the third prediction label includes C second probability values. The category loss value of the consistency constraint can be determined based on the C first probability values ​​and the C second probability values. The category loss value of the consistency constraint refers to the consistency constraint between the prediction label of the initial management model and the prediction label of the initial learning model. A first probability distribution of the coordinate point offset corresponding to the second prediction frame can be determined, and a second probability distribution of the coordinate point offset corresponding to the third prediction frame can be determined. Based on the first probability distribution and the second probability distribution, the coordinate frame loss value of the consistency constraint is determined. The coordinate frame loss value of the consistency constraint refers to the consistency constraint between the prediction frame of the initial management model and the prediction frame of the initial learning model. The second loss value can be determined based on the category loss value and the coordinate frame loss value.

[0100] In a possible implementation, a relative entropy between the first probability distribution and the second probability distribution may be determined, and a coordinate box loss value of the consistency constraint may be determined based on the relative entropy.

[0101] For example, the uncertain pseudo-label corresponds to the second predicted label and the second predicted frame, the third predicted label and the third predicted frame, and the category loss value is determined based on the second predicted label and the third predicted label, and the coordinate frame loss value is determined based on the second predicted frame and the third predicted frame. Then, the second loss value is determined based on the category loss value and the coordinate frame loss value. For example, the second loss value is determined using the following formula: L = L loc +L cls , L is used to represent the second loss value, L loc Used to represent the coordinate frame loss value, L cls Used to represent the class loss value.

[0102] Among them, for the category loss value L cls For example, the second prediction label includes C first probability values ​​corresponding to C categories, and the third prediction label includes C second probability values ​​corresponding to C categories. Based on the C first probability values ​​and the C second probability values, the category loss value of the consistency constraint can be calculated using the following formula. Of course, the following formula is just an example and there is no limitation to this, as long as the category loss value of the consistency constraint can be determined based on the C first probability values ​​and the C second probability values.

[0103]

[0104] In the above formula, L c-cls represents the category loss value of the consistency constraint, C represents the C categories supported by the initial learning model or the initial management model, i represents the i-th category in the C categories, and p ti represents the second probability value (belonging to the third prediction label) corresponding to the i-th category output by the initial management model. In practical applications, p ti It can be the probability value after sharpening the second probability value corresponding to the i-th category, p si Represents the first probability value corresponding to the i-th category output by the initial learning model (belonging to the second predicted label).

[0105] Among them, the value range of i is 1-C. When i is 1, it means the first category supported by the initial learning model, p ti Indicates the second probability value corresponding to the first category, p si Indicates the first probability value corresponding to the first category. Similarly, when i is C, it indicates the Cth category supported by the initial learning model, p ti Represents the second probability value corresponding to the Cth category, p si Represents the first probability value corresponding to the Cth category.

[0106] Among them, for the coordinate frame loss value l locFor example, the degree of closeness between the coordinates of the four vertices of the second prediction frame and the coordinates of the four vertices of the third prediction frame can be determined. If the degree of closeness indicates that the closer the second prediction frame is to the third prediction frame, the smaller the coordinate frame loss value is. If the degree of closeness indicates that the difference between the second prediction frame and the third prediction frame is greater, the coordinate frame loss value is greater. For example, the first probability distribution of the coordinate point offset corresponding to the four vertex coordinates of the second prediction frame is determined, that is, each vertex coordinate corresponds to a first probability distribution, and the second probability distribution of the coordinate point offset corresponding to the four vertex coordinates of the third prediction frame is determined, that is, each vertex coordinate corresponds to a second probability distribution. When determining the probability distribution corresponding to the vertex coordinates, the probability distribution is represented by the mean and variance. For example, the probability distribution corresponding to the vertex coordinate x is N(μ tx ,∑ tx ).

[0107] Based on the first probability distribution and the second probability distribution, the coordinate frame loss value of the consistency constraint can be determined. For example, the coordinate frame loss value of the consistency constraint can be calculated using the following formula. Of course, the following formula is just an example and there is no restriction on it, as long as the coordinate frame loss value can be obtained.

[0108]

[0109] In the above formula, L c-loc Represents the coordinate box loss value of the consistency constraint, a represents the four vertex coordinates of the predicted box, such as a can be the vertex coordinate (x, y), a can be the vertex coordinate (x+w, y), a can be the vertex coordinate (x, y+h), a can be the vertex coordinate (x+w, y+h), that is, a can be four vertex coordinates.

[0110] KL stands for KL (Kullback-Leibler) divergence, which can also be called relative entropy or information divergence, etc. ta-s ,∑ ta-s ) represents the first probability distribution (also called Gaussian distribution) of the coordinate point offset corresponding to the vertex coordinate a, that is, the probability distribution corresponding to the second prediction box output by the initial learning model, N(μ ta-t ,∑ ta-t ) represents the second probability distribution of the coordinate point offset corresponding to the vertex coordinate a, that is, the probability distribution corresponding to the third prediction box output by the initial management model. In practical applications, ∑ ta-t Sharpening is also possible.

[0111] In summary, we can calculate the relative entropy between the first probability distribution of the coordinate point offset corresponding to the vertex coordinate (x, y) and the second probability distribution of the coordinate point offset corresponding to the vertex coordinate (x, y), and calculate the relative entropy between the first probability distribution of the coordinate point offset corresponding to the vertex coordinate (x+w, y) and the second probability distribution of the coordinate point offset corresponding to the vertex coordinate (x+w, y), and calculate the relative entropy between the first probability distribution of the coordinate point offset corresponding to the vertex coordinate (x, y+h) and the second probability distribution of the coordinate point offset corresponding to the vertex coordinate (x, y+h), and calculate the relative entropy between the first probability distribution of the coordinate point offset corresponding to the vertex coordinate (x+w, y+h) and the second probability distribution of the coordinate point offset corresponding to the vertex coordinate (x+w, y+h). Then, calculate the sum of the above four relative entropies, which is the coordinate box loss value L of the consistency constraint. c-loc .

[0112] To sum up, the category loss value and coordinate frame loss value corresponding to the uncertain pseudo label can be obtained, and the second loss value corresponding to the uncertain pseudo label can be determined based on the category loss value and the coordinate frame loss value.

[0113] Step 209: Determine a target loss value for the initial learning model based on the first loss value, the second loss value, and the third loss value. For example, the average of the first loss value, the second loss value, and the third loss value can be used as the target loss value for the initial learning model, or the sum of the first loss value, the second loss value, and the third loss value can be used as the target loss value for the initial learning model. There is no limitation to this.

[0114] Step 210: Adjust the network parameters of the initial learning model based on the target loss value to obtain an adjusted learning model. For example, after obtaining the target loss value of the initial learning model, the network parameters of the initial learning model can be adjusted based on the target loss value, such as by using a gradient descent method. This embodiment does not limit this adjustment process.

[0115] Step 211: Adjust the network parameters of the initial management model based on the network parameters of the adjusted learning model to obtain the adjusted management model. For example, after obtaining the adjusted learning model, parameter correction values ​​of the network parameters are determined based on the network parameters of the adjusted learning model and the configured proportional coefficients. The network parameters of the initial management model are adjusted based on the parameter correction values ​​to obtain the adjusted management model.

[0116] For example, based on the network parameters of the adjusted learning model, an EMA (Exponential Moving Average) algorithm can be used to determine the network parameters of the management model, thereby obtaining the adjusted management model. When using the EMA algorithm to determine the network parameters of the management model, parameter correction values ​​can be determined based on the network parameters of the adjusted learning model and the proportional coefficient, and the network parameters of the initial management model can be adjusted based on the parameter correction values ​​to obtain the adjusted management model. This process is not limited.

[0117] Step 212: Determine whether the adjusted management model has converged. If the adjusted management model has not converged, then proceed to step 213; if the adjusted management model has converged, then proceed to step 214.

[0118] For example, if the number of iterations of the initial management model or initial learning model reaches a threshold, the adjusted management model is determined to have converged; if the number of iterations of the initial management model or initial learning model does not reach the threshold, the adjusted management model is determined to have not converged. Alternatively, if the target loss value of the initial learning model is less than a loss value threshold, the adjusted management model is determined to have converged; if the target loss value of the initial learning model is not less than the loss value threshold, the adjusted management model is determined to have not converged.

[0119] Step 213 : If the adjusted management model has not converged, the adjusted learning model is determined as the initial learning model, the adjusted management model is determined as the initial management model, and the process returns to step 203 .

[0120] Step 214: If the adjusted management model has converged, the converged adjusted management model is determined as the target management model, and the target management model is the model that needs to be output eventually.

[0121] For example, after obtaining the target management model, target detection can be performed on the data to be detected based on the target management model. For example, the data to be detected (such as an image to be detected) can be input into the target management model, and the target management model outputs a target frame in the data to be detected and identifies the target or target category, such as recognizing a human face, recognizing a vehicle category, recognizing an animal category, recognizing an electronic product category, etc. This process will not be described in detail.

[0122] It can be seen from the above technical solutions that in the embodiment of the present application, a target management model can be obtained by training with a small amount of labeled data and a large amount of unlabeled data, thereby avoiding the acquisition of a large amount of labeled data, reducing the workload of the annotation operation, saving human resources, and the target management model has relatively good performance and high reliability. On the basis of effective use of pseudo labels, the robustness to noise samples in the training process is improved, and the target management model has a very large improvement compared to the baseline model. Setting pseudo labels of different proportions to high-quality pseudo labels can achieve relatively good training results, have good robustness to noise samples, and are insensitive to hyperparameters. Based on the principle that the features of the same image after different augmentations remain consistent, consistency constraints are established for the prediction boxes and prediction labels under different augmentations of unlabeled data, which can improve the robustness to noise samples in the training process on the basis of effective use of pseudo labels. Pseudo labels and consistency constraints can be combined, and pseudo labels are used as true categories for reliable prediction boxes. For uncertain prediction boxes, consistency comparison losses are established for the prediction results (or features), and the prediction results (or features) of different views are used to constrain the same target to not change its category characteristics under different augmentations. A management model-learning model is used to produce a smoother and more stable classifier, and only the gradient of the learning model is returned.

[0123] Based on the same application concept as the above method, a target detection device is proposed in the embodiment of the present application, see Figure 3 FIG. 1 is a schematic diagram of the structure of the target detection device, which may include:

[0124] An acquisition module 31 is used to acquire an initial management model and an initial learning model, add pseudo labels to unlabeled data based on the initial management model, and divide the pseudo labels into high-quality pseudo labels and uncertain pseudo labels; a determination module 32 is used to input the unlabeled data into the initial learning model to obtain a first prediction value corresponding to the unlabeled data; determine a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label, and determine a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label; input the unlabeled data into the initial management model to obtain a second prediction value corresponding to the unlabeled data, and determine a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label; a processing module 33 is used to train the initial management model based on the first prediction label, the first prediction box, the second prediction label, the second prediction box, the third prediction label and the third prediction box to obtain a trained target management model; wherein, the target management model is used to perform target detection on the data to be detected.

[0125] Exemplarily, when the acquisition module 31 acquires the initial management model and the initial learning model, it is specifically used to: obtain a baseline model using labeled data training; generate an initial management model and an initial learning model based on the baseline model; wherein, the network structure of the initial management model is the same as the network structure of the baseline model, and the network parameters of the initial management model are the same as or different from the network parameters of the baseline model; the network structure of the initial learning model is the same as the network structure of the baseline model, and the network parameters of the initial learning model are the same as or different from the network parameters of the baseline model.

[0126] Exemplarily, the acquisition module 31 adds pseudo-labels to the unlabeled data based on the initial management model, and divides the pseudo-labels into high-quality pseudo-labels and uncertain pseudo-labels, and is specifically used to: for each unlabeled data, input the unlabeled data into the initial management model to obtain the pseudo-label corresponding to the unlabeled data, and the probability value corresponding to the pseudo-label; for each category supported by the initial management model, sort all pseudo-labels corresponding to the category based on the probability values ​​corresponding to all pseudo-labels corresponding to the category; based on the sorting results, select K pseudo-labels with large probability values ​​as high-quality pseudo-labels, and select the remaining pseudo-labels except the high-quality pseudo-labels as uncertain pseudo-labels; wherein K is a positive integer.

[0127] Exemplarily, the processing module 33 trains the initial management model based on the first prediction label, the first prediction box, the second prediction label, the second prediction box, the third prediction label and the third prediction box to obtain the trained target management model, which is specifically used to: determine the first loss value based on the first prediction label and the first prediction box; determine the second loss value based on the second prediction label, the second prediction box, the third prediction label and the third prediction box; train the initial management model based on the first loss value and the second loss value to obtain the target management model.

[0128] Exemplarily, when the processing module 33 determines the second loss value based on the second prediction label, the second prediction box, the third prediction label and the third prediction box, it is specifically used to: if the second prediction label includes C first probability values ​​corresponding to the C categories supported by the initial learning model, and the third prediction label includes C second probability values ​​corresponding to the C categories supported by the initial management model, then determine the category loss value of the consistency constraint based on the C first probability values ​​and the C second probability values; wherein, C is a positive integer greater than 1; determine the first probability distribution of the coordinate point offset corresponding to the second prediction box, and determine the second probability distribution of the coordinate point offset corresponding to the third prediction box, and determine the coordinate box loss value of the consistency constraint based on the first probability distribution and the second probability distribution; determine the second loss value based on the category loss value and the coordinate box loss value.

[0129] Exemplarily, when the processing module 33 determines the coordinate frame loss value of the consistency constraint based on the first probability distribution and the second probability distribution, it is specifically used to: determine the relative entropy between the first probability distribution and the second probability distribution; and determine the coordinate frame loss value of the consistency constraint based on the relative entropy.

[0130] Exemplarily, the processing module 33 trains the initial management model based on the first loss value and the second loss value, and when obtaining the target management model, is specifically used to: adjust the network parameters of the initial learning model based on the first loss value and the second loss value to obtain an adjusted learning model; adjust the network parameters of the initial management model based on the network parameters of the adjusted learning model to obtain an adjusted management model; if the adjusted management model has not converged, determine the adjusted learning model as the initial learning model, determine the adjusted management model as the initial management model, and return to execute the operation of adding pseudo labels to unlabeled data based on the initial management model, and dividing the pseudo labels into high-quality pseudo labels and uncertain pseudo labels; if the adjusted management model has converged, determine the adjusted management model as the target management model.

[0131] Exemplarily, the processing module 33 adjusts the network parameters of the initial management model based on the network parameters of the adjusted learning model, and when obtaining the adjusted management model, is specifically used to: determine the parameter correction values ​​of the network parameters based on the network parameters of the adjusted learning model and the configured proportional coefficients, and adjust the network parameters of the initial management model based on the parameter correction values ​​to obtain the adjusted management model.

[0132] Based on the same application concept as the above method, a target detection device is proposed in the embodiment of the present application, see Figure 4As shown, the target detection device may include: a processor 41 and a machine-readable storage medium 42, wherein the machine-readable storage medium 42 stores machine-executable instructions that can be executed by the processor 41; the processor 41 is configured to execute the machine-executable instructions to implement the target detection method disclosed in the above example of this application. For example, the processor 41 is configured to execute the machine-executable instructions to implement the following steps:

[0133] Obtaining an initial management model and an initial learning model, adding pseudo labels to unlabeled data based on the initial management model, and dividing the pseudo labels into high-quality pseudo labels and uncertain pseudo labels;

[0134] Inputting unlabeled data into the initial learning model to obtain a first prediction value corresponding to the unlabeled data; determining a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label, and determining a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label;

[0135] Inputting the unlabeled data into the initial management model to obtain a second prediction value corresponding to the unlabeled data, and determining a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label;

[0136] Training an initial management model based on the first prediction label, the first prediction frame, the second prediction label, the second prediction frame, the third prediction label, and the third prediction frame to obtain a trained target management model;

[0137] The target management model is used to perform target detection on the data to be detected.

[0138] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the target detection method disclosed in the above example of the present application can be implemented.

[0139] The machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0140] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0141] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0142] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0143] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] Furthermore, these computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0146] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire an initial management model and an initial learning model, add pseudo labels to unlabeled images based on the initial management model, and divide the pseudo labels into high-quality pseudo labels and uncertain pseudo labels; Inputting the unlabeled image into the initial learning model to obtain a first prediction value corresponding to the unlabeled image; determining a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label, and determining a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label; Inputting the unlabeled image into the initial management model, obtaining a second prediction value corresponding to the unlabeled image, and determining a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label; Training an initial management model based on the first prediction label, the first prediction frame, the second prediction label, the second prediction frame, the third prediction label, and the third prediction frame to obtain a trained target management model; The target management model is used to perform target detection on the image to be detected.

2. The method according to claim 1, characterized in that The obtaining of the initial management model and the initial learning model includes: A baseline model is trained using labeled images; generating an initial management model and an initial learning model based on the baseline model; The network structure of the initial management model is the same as the network structure of the baseline model, and the network parameters of the initial management model are the same as or different from the network parameters of the baseline model; The network structure of the initial learning model is the same as the network structure of the baseline model, and the network parameters of the initial learning model are the same as or different from the network parameters of the baseline model.

3. The method according to claim 1, characterized in that Adding pseudo labels to unlabeled images based on the initial management model and dividing the pseudo labels into high-quality pseudo labels and uncertain pseudo labels, including: For each unlabeled image, input the unlabeled image into the initial management model to obtain a pseudo label corresponding to the unlabeled image and a probability value corresponding to the pseudo label; For each category supported by the initial management model, all pseudo-labels corresponding to the category are sorted based on the probability values ​​corresponding to all pseudo-labels corresponding to the category; based on the sorting results, K pseudo-labels with large probability values ​​are selected as high-quality pseudo-labels, and the remaining pseudo-labels except the high-quality pseudo-labels are determined as uncertain pseudo-labels; wherein K is a positive integer.

4. The method according to claim 1, wherein Inputting the unlabeled image into the initial learning model includes: performing a first data augmentation on the unlabeled image, and inputting the unlabeled image after the first data augmentation into the initial learning model; Inputting the unlabeled image to the initial management model includes: performing a second data augmentation on the unlabeled image, and inputting the unlabeled image after the second data augmentation to the initial management model; The first data augmentation method is different from the second data augmentation method.

5. The method according to claim 1, characterized in that The training of the initial management model based on the first prediction label, the first prediction frame, the second prediction label, the second prediction frame, the third prediction label, and the third prediction frame to obtain a trained target management model includes: Determine a first loss value based on the first prediction label and the first prediction box; determine a second loss value based on the second prediction label, the second prediction box, the third prediction label and the third prediction box; train the initial management model based on the first loss value and the second loss value to obtain the target management model.

6. The method according to claim 5, characterized in that The determining a second loss value based on the second predicted label, the second predicted box, the third predicted label, and the third predicted box includes: If the second predicted label includes C first probability values ​​corresponding to the C categories supported by the initial learning model, and the third predicted label includes C second probability values ​​corresponding to the C categories supported by the initial management model, then determining a category loss value for the consistency constraint based on the C first probability values ​​and the C second probability values; wherein C is a positive integer greater than 1; Determining a first probability distribution of coordinate point offsets corresponding to the second prediction box, determining a second probability distribution of coordinate point offsets corresponding to the third prediction box, and determining a coordinate box loss value of the consistency constraint based on the first probability distribution and the second probability distribution; A second loss value is determined based on the category loss value and the coordinate box loss value.

7. The method according to claim 5, characterized in that Before training the initial management model based on the first loss value and the second loss value to obtain the target management model, the method further includes: inputting the labeled image into the initial learning model to obtain a third prediction value corresponding to the labeled image, determining a fourth prediction label and a fourth prediction box based on the third prediction value, and determining a third loss value based on the fourth prediction label and the fourth prediction box; The training of the initial management model based on the first loss value and the second loss value to obtain the target management model includes: training the initial management model based on the first loss value, the second loss value and the third loss value to obtain the target management model.

8. The method according to claim 5 or 7, characterized in that The training of the initial management model based on the first loss value and the second loss value to obtain the target management model includes: Adjusting the network parameters of the initial learning model based on the first loss value and the second loss value to obtain an adjusted learning model; adjusting the network parameters of the initial management model based on the network parameters of the adjusted learning model to obtain an adjusted management model; If the adjusted management model does not converge, the adjusted learning model is determined as the initial learning model, the adjusted management model is determined as the initial management model, and the operation of adding pseudo labels to the unlabeled images based on the initial management model and dividing the pseudo labels into high-quality pseudo labels and uncertain pseudo labels is returned; If the adjusted management model has converged, the adjusted management model is determined as the target management model.

9. The method according to claim 8, characterized in that The network parameters of the initial management model are adjusted based on the network parameters of the adjusted learning model to obtain the adjusted management model, including: determining the parameter correction values ​​of the network parameters based on the network parameters of the adjusted learning model and the configured proportional coefficients, and adjusting the network parameters of the initial management model based on the parameter correction values ​​to obtain the adjusted management model.

10. A target detection device, characterized in that: The device comprises: an acquisition module, configured to acquire an initial management model and an initial learning model, add pseudo labels to unlabeled images based on the initial management model, and classify the pseudo labels into high-quality pseudo labels and uncertain pseudo labels; A determination module is configured to input an unlabeled image into an initial learning model to obtain a first prediction value corresponding to the unlabeled image; determine a first prediction label and a first prediction box based on the first prediction value corresponding to the high-quality pseudo label, and determine a second prediction label and a second prediction box based on the first prediction value corresponding to the uncertain pseudo label; input the unlabeled image into an initial management model to obtain a second prediction value corresponding to the unlabeled image, and determine a third prediction label and a third prediction box based on the second prediction value corresponding to the uncertain pseudo label; A processing module is used to train the initial management model based on the first prediction label, the first prediction box, the second prediction label, the second prediction box, the third prediction label and the third prediction box to obtain a trained target management model; wherein the target management model is used to perform target detection on the image to be detected.

Images