Pseudo tag generation method and device

Abstract

The invention provides a pseudo label generation method and device. The method comprises the following steps: pre-training a target detection model by using a labeled first data set; using a pre-trained target detection model to perform reasoning on the second data set with the label under a plurality of preset NMSIOU thresholds to obtain a plurality of groups of reasoning results; the NMSIOU threshold value is an overlapping degree threshold value adopted when the target detection model executes an NMS algorithm in the reasoning process; according to a preset GPIOU threshold value and the multiple groups of reasoning results, positive and negative sample distinguishing indexes and an optimal NMSIOU threshold value in the multiple preset NMSIOU threshold values are determined; the GPIOU threshold value is an overlapping degree threshold value set for the reasoning result and labeling information corresponding to the reasoning result in the second data set; reasoning a third data set homologous with the second data set under the optimal NMSIOU threshold value by using a pre-trained target detection model to obtain a group of reasoning results corresponding to the optimal NMSIOU threshold value, and generating a pseudo tag according to the positive and negative sample distinguishing index and the group of reasoning results.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a pseudo-label generation method and device. Background Art

[0002] When doing target detection, effectively labeling the dataset is a recognized problem and the most important issue in the entire algorithm implementation process. Different target detection tasks require different dataset scenarios, huge data volumes, and high labeling costs. Therefore, it is very important to quickly generate effective pseudo-labels.

[0003] Currently, pseudo labels are used in semi-supervised training. The core of the semi-supervised target detection method is to use the pre-trained model learned on the labeled data set to infer the unlabeled data set, reduce a large number of redundant frames after NMS, use the set threshold to distinguish pseudo labels, and finally use the pseudo labels to train the model, so as to achieve the purpose of using a large amount of unlabeled data to improve the performance of the model on the test set. In the process of semi-supervised training, in order to make the model more accurate when pseudo-labeling on the unlabeled data set, it is necessary to continuously iterate the training, which is time-consuming, and requires a high number of graphics cards, which is too costly. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a pseudo label generation method and device, which can quickly generate pseudo labels with low implementation cost.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A pseudo-label generation method, the method comprising:

[0007] Pre-training the target detection model using the labeled first dataset;

[0008] For each NMS_IOU threshold among a plurality of preset NMS_IOU thresholds, use a pre-trained target detection model to perform reasoning on the annotated second data set under the NMS_IOU threshold to obtain a set of reasoning results corresponding to the NMS_IOU threshold; the NMS_IOU threshold is an overlap threshold used by the target detection model when executing the NMS algorithm during the reasoning process;

[0009] Determine the positive and negative sample discrimination index and the optimal NMS_IOU threshold among the multiple preset NMS_IOU thresholds according to the preset GP_IOU threshold, the annotation information of the second data set, and the multiple groups of inference results corresponding to the multiple preset NMS_IOU thresholds; the GP_IOU threshold is an overlap threshold set for the inference result and the annotation information corresponding to the inference result in the second data set;

[0010] Using the pre-trained target detection model, a third data set homologous to the second data set is inferred under the optimal NMS_IOU threshold to obtain a set of inference results corresponding to the optimal NMS_IOU threshold, and a pseudo label is generated according to the positive and negative sample discrimination index and the set of inference results.

[0011] A pseudo-label generating device, comprising:

[0012] A training unit, used for pre-training a target detection model using a labeled first data set;

[0013] A first reasoning unit is configured to use a pre-trained target detection model to perform reasoning on a second labeled data set under the NMS_IOU threshold for each of a plurality of preset NMS_IOU thresholds, and obtain a set of reasoning results corresponding to the NMS_IOU threshold; the NMS_IOU threshold is an overlap threshold used by the target detection model when executing the NMS algorithm during the reasoning process;

[0014] A determination unit, configured to determine a positive and negative sample discrimination index and an optimal NMS_IOU threshold among the plurality of preset NMS_IOU thresholds according to a preset GP_IOU threshold, the annotation information of the second data set, and the plurality of groups of inference results corresponding to the plurality of preset NMS_IOU thresholds; the GP_IOU threshold is an overlap threshold set for the inference result and the annotation information corresponding to the inference result in the second data set;

[0015] The second inference unit is used to use the pre-trained target detection model to infer a third data set that is homologous to the second data set under the optimal NMS_IOU threshold, obtain a set of inference results corresponding to the optimal NMS_IOU threshold, and generate a pseudo label according to the positive and negative sample discrimination index and the set of inference results.

[0016] An electronic device comprises: at least one processor, and a memory connected to the at least one processor via a bus; the memory stores one or more computer programs executable by the at least one processor; and the at least one processor implements the steps in the above-mentioned pseudo-label generation method when executing the one or more computer programs.

[0017] A computer-readable storage medium stores one or more computer programs, and the one or more computer programs are executed by a processor to perform the steps in the pseudo-label generation method.

[0018] It can be seen from the above technical solution that in the present invention, the target detection model is first pre-trained using the labeled first data set, and then the pre-trained target detection model is used to infer the labeled second data set under multiple preset NMS_IOU thresholds to obtain multiple groups of inference results, and then the positive and negative sample discrimination index and the optimal NMS_IOU threshold among the multiple preset NMS_IOU thresholds are determined according to the preset GP_IOU threshold, the annotation information of the second data set, and the multiple groups of inference results. In this way, the pre-trained target detection model can be used to infer the unlabeled third data set under the optimal NMS_IOU threshold, and a pseudo label is generated according to the positive and negative sample discrimination index and the corresponding inference results. The whole process does not require iterative training, is less time-consuming, does not require the number of graphics cards, and has a low implementation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 is a flow chart of a pseudo-label generation method according to a first embodiment of the present invention;

[0021] Figure 2 is a flow chart of a pseudo-label generation method according to a second embodiment of the present invention;

[0022] Figure 3 is a flow chart of a method for determining a performance evaluation result and an optimal NMS_IOU threshold of a target detection model according to an embodiment of the present invention;

[0023] Figure 4 is an example diagram of performance evaluation data of an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of the structure of a pseudo-label generating device according to an embodiment of the present invention;

[0025] Figure 6 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0028] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0029] See also Figure 1 , Figure 1 is a flow chart of a pseudo-label generation method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0030] S101, pre-training a target detection model using a labeled first data set;

[0031] In an embodiment of the present invention, the first data set is a training set, and the data in the first data set may be image data with detection targets labeled, and the labeling information of each image data includes the labeling information of each detection target in the image data, and the labeling information of each detection target may be the target box where the detection target is located in the image data. Each image data in the first data set and the labeling information of the image data may be input as a training sample into a pre-built target detection model to pre-train the pre-built target detection model.

[0032] S102, for each NMS_IOU threshold among a plurality of preset NMS_IOU thresholds, use a pre-trained target detection model to perform reasoning on the annotated second data set under the NMS_IOU threshold to obtain a set of reasoning results corresponding to the NMS_IOU threshold; the NMS_IOU threshold is an overlap threshold used by the target detection model when executing the NMS algorithm during the reasoning process;

[0033] In an embodiment of the present invention, the second data set is a verification set, and the data in the second data set may be annotated image data, and the annotation information of each image data includes the annotation information of each detection target in the image data, and the annotation information of each detection target may be the target box where the detection target is located in the image data. When reasoning is performed on the annotated second data set at each NMS_IOU threshold, each image data in the second data set (excluding the annotation information of the image data) may be input into a pre-trained target detection model, and reasoning may be performed at the NMS_IOU threshold to obtain one or more reasoning results at the NMS_IOU threshold corresponding to the detection target in the image data, and the one or more reasoning results are the predicted box of the detection target in the image data, and the predicted box of the detection target may be verified using the target box where the detection target is located in the image data.

[0034] In the embodiment of the present invention, a set of inference results consisting of inference results corresponding to the detection targets in all the image data in the second data set at each NMS_IOU threshold is called a set of inference results corresponding to the NMS_IOU threshold.

[0035] S103, determining a positive and negative sample discrimination index and an optimal NMS_IOU threshold among the multiple NMS_IOU thresholds according to a preset GP_IOU threshold, the annotation information of the second data set, and multiple groups of inference results corresponding to the multiple NMS_IOU thresholds; the GP_IOU threshold is an overlap threshold set for the inference result and the annotation information corresponding to the inference result in the second data set;

[0036] In an embodiment of the present invention, the annotation information of the second data set includes the annotation information of each image data in the second data set. The multiple groups of inference results include a group of inference results corresponding to each NMS_IOU threshold.

[0037] In an embodiment of the present invention, when generating a pseudo label for an inference result, it is necessary to distinguish whether the inference result is a positive sample or a negative sample. When the inference result is a positive sample, a pseudo label indicating that the inference result is a positive sample can be generated. When the inference result is a negative sample, a pseudo label indicating that the inference result is a negative sample can be generated.

[0038] In an embodiment of the present invention, the positive-negative sample distinction index is a measurement index used to measure whether an inference result is a positive sample or a negative sample, including a first confidence threshold and a second confidence threshold, the first confidence threshold is less than the second confidence threshold, the first confidence threshold is used to measure whether an inference result is a negative sample, and the second confidence threshold is used to measure whether an inference result is a positive sample.

[0039] S104. Use the pre-trained target detection model to infer a third data set that is homologous to the second data set under the optimal NMS_IOU threshold to obtain a set of inference results corresponding to the optimal NMS_IOU threshold, and generate a pseudo label based on the positive and negative sample discrimination index and the set of inference results.

[0040] In an embodiment of the present invention, the second data set and the third data set are homologous data sets, that is, the data in the second data set and the data in the third data set are all from the same batch of data, which can be obtained from the same scene or similar scenes. For example, it can be a batch of image data taken by a surveillance camera installed at a certain intersection.

[0041] In an embodiment of the present invention, a pre-trained target detection model is used to perform inference on the third data set under the optimal NMS_IOU threshold. After obtaining a set of inference results corresponding to the optimal NMS_IOU threshold, it can be determined whether each inference result is a positive sample or a negative sample according to the positive and negative sample distinction index. For each inference result determined to be a positive sample, a pseudo label indicating that the inference result is a positive sample can be generated. For each inference result determined to be a negative sample, a pseudo label indicating that the inference result is a negative sample can be generated.

[0042] from Figure 1 It can be seen from the method shown that in the present invention, the target detection model is first pre-trained using the labeled first data set, and then the pre-trained target detection model is used to infer the labeled second data set under multiple preset NMS_IOU thresholds to obtain multiple groups of inference results, and then the positive and negative sample discrimination index and the optimal NMS_IOU threshold among the multiple NMS_IOU thresholds are determined according to the preset GP_IOU threshold and the multiple groups of inference results, so that the pre-trained target detection model can be used to infer the unlabeled third data set under the optimal NMS_IOU threshold, and a pseudo label is generated according to the positive and negative sample discrimination index and the corresponding inference results. The whole process does not require iterative training, is less time-consuming, does not require the number of graphics cards, and has a low implementation cost.

[0043] See also Figure 2 , Figure 2 is a flow chart of a pseudo-label generation method according to a second embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0044] S201, pre-training a target detection model using a labeled first data set;

[0045] In an embodiment of the present invention, the data in the first data set may be image data with detection targets already labeled, and the labeling information of each image data includes the labeling information of each detection target in the image data, and the labeling information of each detection target may be the target box where the detection target is located in the image data. By training these labeled image data, a pre-trained target detection model may be obtained.

[0046] S202, for each NMS_IOU threshold among a plurality of preset NMS_IOU thresholds, use a pre-trained target detection model to perform reasoning on the second data set with annotations under the NMS_IOU threshold, and obtain a set of reasoning results corresponding to the NMS_IOU threshold; the NMS_IOU threshold is an overlap threshold used by the target detection model when executing the NMS algorithm during the reasoning process;

[0047] In an embodiment of the present invention, the data in the second data set may be image data with annotation information, and the annotation information of each image data includes annotation information of each detection target in the image data, and the annotation information of each detection target may be a target box in the image data where the detection target is located.

[0048] In practical applications, when using a pre-trained target detection model to infer each image data, multiple prediction boxes corresponding to the detection target in the image data can be obtained. The non-maximum suppression (NMS) algorithm can be used to remove redundant prediction boxes with high overlap (also known as intersection-over-union ratio). Each remaining prediction box is an inference result corresponding to the detection target in the image data.

[0049] In an embodiment of the present invention, the IOU threshold used when executing the NMS algorithm is called the NMS_IOU threshold. Multiple NMS_IOU thresholds can be set in advance. Using a pre-trained target detection model, the labeled second data set is inferred at each NMS_IOU threshold to obtain a set of inference results corresponding to the NMS_IOU threshold.

[0050] S2031. Calculate the overlap between each inference result in a group of inference results corresponding to each NMS_IOU threshold and the annotation information corresponding to the inference result in the second data set, and determine whether the inference result is a positive sample or a negative sample according to the overlap and a preset GP_IOU threshold.

[0051] In an embodiment of the present invention, an overlap threshold (hereinafter referred to as GP_IOU threshold) can be set in advance for the inference result and the annotation information corresponding to the inference result in the second data set as a standard for dividing each inference result obtained by inferring the second data set into positive samples and negative samples. Specifically, the inference result (i.e., the prediction box) whose overlap with the annotation information (i.e., the target box) of the detection target in the image data is greater than the GP_IOU threshold can be determined as a positive sample, and the inference result whose overlap with the annotation information of the detection target in the image data is not greater than the GP_IOU threshold can be determined as a negative sample.

[0052] In an embodiment of the present invention, when a pre-trained target detection model is used to infer an image data in the second data set, multiple inference results (i.e., multiple prediction boxes) corresponding to the detection target in the image data can be obtained, and the annotation information corresponding to these inference results is the annotation information of the detection target in the image data (i.e., the target box where the detection target is located). The overlap between each prediction box and the target box where the detection target is located in the image data can be calculated, and the prediction box can be determined to be a positive sample or a negative sample accordingly. For example, specifically, when the overlap is greater than a preset GP_IOU threshold, the prediction box is determined to be a positive sample, otherwise, the prediction box is determined to be a negative sample.

[0053] S2032. Determine the performance evaluation result of the pre-trained target detection model under the confidence level of each inference result in a set of inference results corresponding to each NMS_IOU threshold and the optimal NMS_IOU threshold among the different NMS_IOU thresholds.

[0054] In the embodiment of the present invention, a pre-trained target detection model is used to infer each image data in the second data set, and each obtained inference result corresponds to a confidence level.

[0055] In the embodiment of the present invention, the performance evaluation result of the pre-trained target detection model under the confidence level of each inference result in a set of inference results corresponding to each NMS_IOU threshold and the optimal NMS_IOU threshold among the different NMS_IOU thresholds are determined. Specifically, Figure 3 The method shown implements:

[0056] S2032a, determining the recall and precision of a group of inference results corresponding to each NMS_IOU threshold at the confidence level of each inference result in the group of inference results;

[0057] In an embodiment of the present invention, the group of inference results can be sorted from high to low according to the confidence, and then the recall rate and precision can be calculated based on the confidence of each inference result in the sorting. Specifically, the recall rate and precision of the first n inference results in the sorting results can be calculated, and the recall rate and precision are determined as the recall rate and precision of the pre-trained target detection model at the confidence of the nth inference result.

[0058] In the embodiment of the present invention, after determining the recall rate and precision of each group of inference results at the confidence level of each inference result in the group of inference results, the recall rate and precision of each inference result in the group of inference results at the confidence level can be updated according to the number of data in the first data set and the number of data in the second data set, thereby improving the accuracy of the recall rate and precision calculation.

[0059] The precision can be updated according to the following formula:

[0060] P new =(ratio+(1-ratio)×T all ×R old ) / (ratio×T all +(1-ratio)×T all ×R old / P old );

[0061] The recall can be updated according to the following formula:

[0062] R new =ratio+(1-ratio)×R old ;

[0063] In the above formulas for calculating precision and recall, P old and R old are the precision and recall before updating, P new and R new are the updated precision and recall respectively; T all =T1+T2, T1 is the number of data in the first data set, T2 is the number of data in the second data set; ratio = T1 / T all .

[0064] S2032b, according to the recall rate and precision of a group of inference results corresponding to each NMS_IOU threshold at the confidence level of each inference result in the group of inference results, determine the detection precision of the pre-trained target detection model under the group of inference results and the performance evaluation result under the confidence level of the inference result in the group of inference results;

[0065] In an embodiment of the present invention, the detection accuracy of the pre-trained target detection model under a group of inference results corresponding to each NMS_IOU threshold is determined according to the recall rate and precision of each inference result in the group of inference results at the confidence level. This can be achieved by various methods. For example, the detection accuracy of the pre-trained target detection model under the group of inference results can be determined by interpolation and integration using the VOC2007 method, the VOC2010 method, etc.

[0066] In the embodiment of the present invention, the performance evaluation result of the pre-trained target detection model under the confidence level of each inference result in a set of inference results corresponding to each NMS_IOU threshold may specifically include two F scores: F β1_score and F β2_score , where β1 and β2 are both pre-set values ​​for measuring the importance of precision and recall. Specifically, β1 can be a value indicating that precision is more important than recall, and β2 can be a value indicating that recall is more important than precision, for example, β1=0.5, β2=2.

[0067] In the embodiment of the present invention, the F score can be calculated using the following formula:

[0068] F β_score =((1+β)×Precision×Recall) / (β×Precision+Recall);

[0069] Where Precision is the accuracy, Recall is the recall rate, and β is a value used to measure the importance of precision and recall rate. For example, when β = 2, the F score is expressed as F 2_score , in this case, recall is twice as important as precision; when β = 0.5, the F score is expressed as F 0.5_score , in this case, precision is twice as important as recall.

[0070] In the embodiment of the present invention, according to the recall rate and precision of a group of inference results corresponding to each NMS_IOU threshold under the confidence level of each inference result in the group of inference results, the performance evaluation result of the pre-trained target detection model under the group of inference results is determined, which can be specifically implemented by the following method:

[0071] The following formula is used to calculate the Fβ1 score of the pre-trained object detection model under this set of inference results:

[0072] F β1_score =((1+β1)×P×R) / (β1×P+R);

[0073] The following formula is used to calculate the Fβ2 score of the pre-trained object detection model under this set of inference results:

[0074] F β2_score =((1+β2)×P×R) / (β2×P+R);

[0075] In the calculation formulas of the above Fβ1 score and Fβ2 score, P and R are the precision and recall of the pre-trained target detection model under this set of inference results, respectively.

[0076] S2032c. Determine the optimal NMS_IOU threshold according to the detection accuracy of the pre-trained target detection model under a set of inference results corresponding to each NMS_IOU threshold.

[0077] In an embodiment of the present invention, determining the optimal NMS_IOU threshold according to the detection accuracy of the pre-trained target detection model under a set of inference results corresponding to each NMS_IOU threshold may specifically include:

[0078] Determine a set of inference results corresponding to the maximum detection accuracy of the pre-trained object detection model;

[0079] The NMS_IOU threshold used when the pre-trained target detection model performs inference on the second data set to obtain the group of inference results is determined as the optimal NMS_IOU threshold.

[0080] S2033. Determine a positive and negative sample discrimination index according to a performance evaluation result of the pre-trained target detection model under the confidence level of each inference result in a set of inference results corresponding to each NMS_IOU threshold.

[0081] In the embodiment of the present invention, the positive and negative sample distinguishing index includes a first confidence threshold for distinguishing negative samples and a second confidence threshold for distinguishing positive samples;

[0082] According to the performance evaluation results of the pre-trained target detection model under the confidence level of each inference result in a set of inference results corresponding to each NMS_IOU threshold, the positive and negative sample discrimination indicators are determined, including:

[0083] Find F in a set of inference results corresponding to the maximum detection accuracy of the pre-trained target detection model. β1_score The confidence score of the inference result corresponding to the performance evaluation result with the largest value β1 ;

[0084] Find F in a set of inference results corresponding to the maximum detection accuracy of the pre-trained target detection model. β2_score The confidence score of the inference result corresponding to the performance evaluation result with the largest value β2 ;

[0085] Score β1 and scoreβ2 The minimum value and the maximum value in are respectively determined as the first confidence threshold and the second confidence threshold.

[0086] The above steps 2031 to 2033 are Figure 1 The specific refinement of step 103 is shown.

[0087] S204. Use the pre-trained target detection model to infer a third data set that is homologous to the second data set under the optimal NMS_IOU threshold to obtain a set of inference results corresponding to the optimal NMS_IOU threshold, and generate a pseudo label based on the positive and negative sample discrimination index and the set of inference results.

[0088] In an embodiment of the present invention, the data in the third data set may be unlabeled image data. The third data set and the second data set are homologous data sets. In practical applications, the number of data in the second data set is generally much smaller than the number of data in the third data set. For example, the number of data in the second data set may be between 5% and 10% of the number of data in the third data set.

[0089] In the embodiment of the present invention, generating a pseudo label according to the positive and negative sample distinguishing index and the group of inference results may specifically include:

[0090] Determine the inference results whose confidence is less than the first confidence threshold in the group of inference results, and generate a pseudo label indicating that the inference result is a negative sample for each determined inference result;

[0091] An inference result having a confidence level greater than a second confidence threshold in the group of inference results is determined, and a pseudo label indicating that the inference result is a positive sample is generated for each determined inference result.

[0092] In the embodiment of the present invention, after the pseudo labels are generated, step S205 may be continued to be performed to complete the training of the target detection model.

[0093] S205: Train the pre-trained target detection model according to the generated pseudo-labels to obtain a trained target detection model.

[0094] Here, after generating the pseudo labels, the generated pseudo labels are used to annotate the image data in the third data set, and then the annotated third data set can be used to train the pre-trained target detection model to obtain a trained target detection model.

[0095] from Figure 2It can be seen from the method shown that in the present invention, the target detection model is first pre-trained using the labeled first data set, and then the pre-trained target detection model is used to infer the labeled second data set under multiple preset NMS_IOU thresholds to obtain multiple groups of inference results, and then the confidence of each inference result in each group of inference results is determined according to the preset GP_IOU threshold, and then the precision and recall rate of the pre-trained target detection model under the confidence of each inference result in each group of inference results are determined, and the detection accuracy of the pre-trained target detection model under the group of inference results is calculated based on the precision and recall rate (such as when GP_IOU=0.5, the AP50 of the pre-trained target detection model under the group of inference results is calculated as the detection accuracy), and the performance evaluation result of the pre-trained target detection model under the confidence of the inference result in the group of inference results is also calculated based on the precision and recall rate: F β1_score and F β2_score Finally, we can get multiple sets of evaluation data corresponding to each set of inference results: precision P, recall rate R, confidence score, F β1_score and F β2_score In this way, we can select a group of inference results with the largest AP50, and then find F in the multiple groups of evaluation data corresponding to this group of inference results. β1_score The score corresponding to the maximum β1 and F β2_score The score corresponding to the maximum β2 , score β1 and score β2 It can be used to divide positive and negative samples, and generate pseudo labels based on the division results of positive and negative samples, so as to continue training the target detection model using the generated pseudo labels.

[0096] The following is an example of the application process of the embodiment of the present invention in combination with actual application scenarios:

[0097] Assume that the target detection model has been pre-trained using a labeled first data set. The first data set may be image data obtained from multiple different scenes, wherein the image data includes a detection target (eg, a person) and the target frame where the detection target is located has been labeled.

[0098] When you need to use a pre-trained object detection model to detect pedestrians at a certain intersection, in order to improve its detection accuracy, you can generate pseudo labels and use the pseudo labels to further train the pre-trained object detection model. The specific process is as follows:

[0099] The first step is to obtain a batch of image data taken by cameras installed at the intersection and / or other similar intersections (assuming it includes 10,000 images), manually mark 1,000 of the images with target boxes where pedestrians are located, put these 1,000 marked images into the second data set, and put the remaining 9,000 unmarked images into the third data set.

[0100] In the second step, the pre-trained target detection model is used to detect the image data in the second dataset at multiple preset NMS_IOU thresholds (such as Figure 4 As shown, reasoning is performed under the following conditions, including: 0.35, 0.40, 0.45, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80), and multiple groups of reasoning results are obtained.

[0101] In the third step, for each group of inference results, determine whether each inference result in the group of inference results is a positive sample or a negative sample based on the preset GP_IOU threshold (assuming it is 0.5) and the annotation information of the second data set.

[0102] Here, for each inference result in the group of inference results, the overlap between the inference result (prediction box) and its corresponding annotation information (target box) is calculated. If the overlap is greater than 0.5, the inference result is determined to be a positive sample, otherwise, the inference result is determined to be a negative sample.

[0103] The fourth step is to sort each group of inference results from high to low according to the confidence level, and determine the precision P and recall R of the group of inference results at each confidence level according to the sorting order, and update P and R according to the number of data in the first data set and the number of data in the second data set.

[0104] Since it has been determined that each inference result in the group of inference results is a positive sample or a negative sample, when a confidence threshold is defined, the precision P and recall R of the group of inference results under the confidence threshold can be determined.

[0105] The fifth step is to determine the detection accuracy AP50 of the pre-trained target detection model under the confidence level of each reasoning result and the performance evaluation results (including F 0.5_score and F 2_score ).

[0106] After the first five steps, we can finally calculate multiple sets of performance evaluation data corresponding to each set of inference results, such as Figure 4 As shown in the figure, each set of performance evaluation data includes five items: precision P, recall rate R, confidence score, F 0.5_score and F 2_score .

[0107] The sixth step is to determine a set of inference results corresponding to the maximum AP50 value, and determine the NMS_IOU threshold used by the pre-trained target detection model to infer the second data set to obtain the set of inference results as the optimal NMS_IOU threshold. At the same time, find F in the set of inference results. 0.5_score The confidence score of the inference result corresponding to the performance evaluation result with the largest value 0.5 , and F 2_score The confidence score2 of the inference result corresponding to the performance evaluation result with the largest value is used to determine the positive and negative sample discrimination index;

[0108] like Figure 4 As shown in Figure 2, the AP50 value (93.6) of the pre-trained target detection model under the inference results obtained when NMS_IOU=0.6 is the highest. Among the multiple evaluation data corresponding to this set of inference results, F 0.5_score The maximum value is 91.78, and the corresponding score 0.5 is 0.8053; F 2_score The maximum value is 90.03, and the corresponding score2 is 0.2966. Therefore, it can be determined that the optimal NMS_IOU threshold is 0.6, and the first confidence threshold and the second confidence threshold in the positive and negative sample distinction index are 0.2966 and 0.8053 respectively.

[0109] In the seventh step, the pre-trained target detection model is used to infer the third data set under the optimal NMS_IOU threshold (NMS_IOU=0.6) to obtain a set of inference results corresponding to the optimal NMS_IOU threshold.

[0110] In the eighth step, for a set of inference results corresponding to the optimal NMS_IOU threshold, each inference result (i.e., prediction box) with a confidence level higher than 0.8053 is determined as a positive sample and a pseudo label indicating that the inference result is a positive sample is generated, and each inference result (i.e., prediction box) with a confidence level lower than 0.2966 is determined as a negative sample and a pseudo label indicating that the inference result is a negative sample is generated.

[0111] Here, for the inference results (i.e., predicted boxes) with confidence between 0.2966 and 0.8053, they can be set to ignore. When training the target detection model, the inference results set to ignore are neither used as positive samples nor negative samples.

[0112] In the ninth step, the pre-trained target detection model is further trained based on the generated pseudo labels to obtain a trained target detection model.

[0113] Here, after the target detection model is trained, the trained target detection model can be used to perform target detection on the pictures taken by the cameras installed at the above intersections.

[0114] The pseudo label generation method of the embodiment of the present invention is described in detail above. The embodiment of the present invention also provides a pseudo label generation device. Figure 5 For detailed explanation:

[0115] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a pseudo-label generating device according to an embodiment of the present invention. Figure 5 As shown, the device comprises:

[0116] A training unit 501 is used to pre-train an object detection model using a labeled first data set;

[0117] The first reasoning unit 502 is used to use a pre-trained target detection model to reason with respect to each NMS_IOU threshold among a plurality of preset NMS_IOU thresholds, and obtain a set of reasoning results corresponding to the NMS_IOU threshold; the NMS_IOU threshold is an overlap threshold used by the target detection model when executing the NMS algorithm during the reasoning process;

[0118] A determination unit 503 is used to determine a positive and negative sample discrimination index and an optimal NMS_IOU threshold among the multiple preset NMS_IOU thresholds according to a preset GP_IOU threshold, the annotation information of the second data set, and multiple groups of inference results corresponding to the multiple preset NMS_IOU thresholds; the GP_IOU threshold is an overlap threshold set for the inference result and the annotation information corresponding to the inference result in the second data set;

[0119] The second reasoning unit 504 is used to use the pre-trained target detection model to infer a third data set that is homologous to the second data set under the optimal NMS_IOU threshold, obtain a set of reasoning results corresponding to the optimal NMS_IOU threshold, and generate a pseudo label based on the positive and negative sample discrimination index and the set of reasoning results.

[0120] Figure 5 In the device shown,

[0121] The determination unit 503 determines the positive and negative sample discrimination index and the optimal NMS_IOU threshold among the multiple preset NMS_IOU thresholds according to the preset GP_IOU threshold, the annotation information of the second data set, and the multiple groups of reasoning results corresponding to the multiple preset NMS_IOU thresholds, including:

[0122] Calculate the overlap between each inference result in each group of inference results and the annotation information corresponding to the inference result in the second data set, and determine whether the inference result is a positive sample or a negative sample according to the overlap and a preset GP_IOU threshold;

[0123] Determine the performance evaluation result of the pre-trained target detection model at the confidence level of each inference result in each group of inference results and the optimal NMS_IOU threshold;

[0124] According to the performance evaluation results of the pre-trained target detection model under the confidence level of each inference result in each group of inference results, the positive and negative sample discrimination index is determined.

[0125] Figure 5 In the device shown,

[0126] The determining unit 503 determines the performance evaluation result of the pre-trained target detection model under the confidence of each inference result in each group of inference results and the optimal NMS_IOU threshold, including:

[0127] Determine the recall and precision of each set of inference results at the confidence level of each inference result in the set of inference results;

[0128] Determine the detection accuracy of the pre-trained target detection model under the group of reasoning results and the performance evaluation result under the confidence level of the reasoning result in the group of reasoning results according to the recall rate and precision of each reasoning result in the group of reasoning results;

[0129] The optimal NMS_IOU threshold is determined according to the detection accuracy of the pre-trained target detection model under each set of reasoning results.

[0130] Figure 5 In the device shown,

[0131] The determination unit 503 further includes: determining the detection accuracy of the pre-trained target detection model under the group of reasoning results and the performance evaluation result under the confidence of the reasoning result in the group of reasoning results according to the recall rate and precision of each reasoning result in the group of reasoning results under the confidence of each reasoning result in the group of reasoning results;

[0132] According to the number of data in the first data set and the number of data in the second data set, the recall rate and the precision of each inference result in the group of inference results at the confidence level are updated.

[0133] Figure 5 In the device shown,

[0134] The determination unit 503 updates the recall and precision of each inference result in the group of inference results at a confidence level according to the number of data in the first data set and the number of data in the second data set, including:

[0135] Update the precision according to the following formula:

[0136] P new =(ratio+(1-ratio)×T all ×R old ) / (ratio×T all +(1-ratio)×T all ×R old / P old );

[0137] Update the recall rate according to the following formula:

[0138] R new =ratio+(1-ratio)×R old ;

[0139] Among them, P old and R old are the precision and recall before updating, P new and R new are the updated precision and recall respectively; T all =T1+T2, T1 is the number of data in the first data set, T2 is the number of data in the second data set; ratio = T1 / T all .

[0140] Figure 5 In the device shown,

[0141] The determination unit 503 determines the optimal NMS_IOU threshold according to the detection accuracy of the pre-trained target detection model under each group of reasoning results, including:

[0142] Determine a set of inference results corresponding to the maximum detection accuracy of the pre-trained object detection model;

[0143] The NMS_IOU threshold used when the pre-trained target detection model performs inference on the second data set to obtain the group of inference results is determined as the optimal NMS_IOU threshold.

[0144] Figure 5 In the device shown,

[0145] The performance evaluation results include F β1_score and F β2_score ;

[0146] The positive and negative sample distinction index includes a first confidence threshold for distinguishing negative samples and a second confidence threshold for distinguishing positive samples;

[0147] The determination unit 503 determines the positive and negative sample discrimination index according to the performance evaluation result of the pre-trained target detection model under the confidence of each inference result in each group of inference results, including:

[0148] Find F in a set of inference results corresponding to the maximum detection accuracy of the pre-trained target detection model. β1_score The confidence score of the inference result corresponding to the performance evaluation result with the largest value β1 ;

[0149] Find F in a set of inference results corresponding to the maximum detection accuracy of the pre-trained target detection model. β2_score The confidence score of the inference result corresponding to the performance evaluation result with the largest value β2 ;

[0150] Score β1 and score β2 The minimum value and the maximum value in are respectively determined as the first confidence threshold and the second confidence threshold.

[0151] Figure 5 In the device shown,

[0152] The second reasoning unit 504 generates a pseudo label according to the positive and negative sample discrimination index and the group of reasoning results, including:

[0153] Determine the inference results whose confidence is less than the first confidence threshold in the group of inference results, and generate a pseudo label indicating that the inference result is a negative sample for each determined inference result;

[0154] An inference result having a confidence level greater than a second confidence threshold in the group of inference results is determined, and a pseudo label indicating that the inference result is a positive sample is generated for each determined inference result.

[0155] Figure 5 In the device shown,

[0156] The training unit 501 is further used to: train the pre-trained target detection model according to the generated pseudo-labels to obtain a trained target detection model.

[0157] The embodiment of the present invention further provides an electronic device, such as Figure 6As shown, the electronic device includes: at least one processor 601, and a memory 602 connected to the at least one processor 601 through a bus; the memory 602 stores one or more computer programs that can be executed by the at least one processor 601; when the at least one processor 601 executes the one or more computer programs, the above Figure 1 or Figure 2 Steps in the pseudo-label generation method shown.

[0158] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more computer programs, and when the one or more computer programs are executed by a processor, the above Figure 1 or Figure 2 Steps in the pseudo-label generation method shown.

[0159] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A pseudo-label generation method, characterized in that: The method includes: Pre-training the target detection model using the labeled first dataset; For each NMS_IOU threshold among a plurality of preset NMS_IOU thresholds, use a pre-trained target detection model to perform reasoning on the annotated second data set under the NMS_IOU threshold to obtain a set of reasoning results corresponding to the NMS_IOU threshold; the NMS_IOU threshold is an overlap threshold used by the target detection model when executing the NMS algorithm during the reasoning process; Determine the positive and negative sample discrimination index and the optimal NMS_IOU threshold among the multiple preset NMS_IOU thresholds according to the preset GP_IOU threshold, the annotation information of the second data set, and the multiple groups of inference results corresponding to the multiple preset NMS_IOU thresholds; the GP_IOU threshold is an overlap threshold set for the inference result and the annotation information corresponding to the inference result in the second data set; Using the pre-trained target detection model, a third data set homologous to the second data set is inferred under the optimal NMS_IOU threshold to obtain a set of inference results corresponding to the optimal NMS_IOU threshold, and a pseudo label is generated according to the positive and negative sample discrimination index and the set of inference results.

2. The method according to claim 1, characterized in that Determining a positive and negative sample discrimination index and an optimal NMS_IOU threshold among the multiple preset NMS_IOU thresholds according to the preset GP_IOU threshold, the annotation information of the second data set, and multiple groups of reasoning results corresponding to the multiple preset NMS_IOU thresholds, including: Calculate the overlap between each inference result in each group of inference results and the annotation information corresponding to the inference result in the second data set, and determine whether the inference result is a positive sample or a negative sample according to the overlap and a preset GP_IOU threshold; Determine the performance evaluation result of the pre-trained target detection model at the confidence level of each inference result in each group of inference results and the optimal NMS_IOU threshold; According to the performance evaluation results of the pre-trained target detection model under the confidence level of each inference result in each group of inference results, the positive and negative sample discrimination index is determined.

3. The method according to claim 2, characterized in that Determining the performance evaluation result of the pre-trained target detection model at the confidence level of each inference result in each group of inference results and the optimal NMS_IOU threshold, including: Determine the recall and precision of each set of inference results at the confidence level of each inference result in the set of inference results; Determine the detection accuracy of the pre-trained target detection model under the group of reasoning results and the performance evaluation result under the confidence level of the reasoning result in the group of reasoning results according to the recall rate and precision of each reasoning result in the group of reasoning results; The optimal NMS_IOU threshold is determined according to the detection accuracy of the pre-trained target detection model under each set of reasoning results.

4. The method according to claim 3, characterized in that Before determining the detection accuracy of the pre-trained target detection model under the group of reasoning results and the performance evaluation result under the confidence level of the reasoning result in the group of reasoning results according to the recall rate and precision of each reasoning result in the group of reasoning results, the method further comprises: According to the number of data in the first data set and the number of data in the second data set, the recall rate and the precision of each inference result in the group of inference results at the confidence level are updated.

5. The method according to claim 4, characterized in that According to the number of data in the first data set and the number of data in the second data set, updating the recall rate and precision of each inference result in the group of inference results at the confidence level, including: Update the precision according to the following formula: P new =(ratio+(1-ratio)×T all ×R old ) / (ratio×T all +(1-ratio)×T all ×R old / P old ); Update the recall rate according to the following formula: R new =ratio+(1-ratio)×R old ; Among them, P old and R old are the precision and recall before updating, P new and R new are the updated precision and recall respectively; T all =T1+T2, T1 is the number of data in the first data set, T2 is the number of data in the second data set; ratio = T1 / T all .

6. The method according to claim 3 or 4, characterized in that: According to the detection accuracy of the pre-trained target detection model under each set of inference results, the optimal NMS_IOU threshold is determined, including: Determine a set of inference results corresponding to the maximum detection accuracy of the pre-trained object detection model; The NMS_IOU threshold used when the pre-trained target detection model performs inference on the second data set to obtain the group of inference results is determined as the optimal NMS_IOU threshold.

7. The method according to claim 3 or 4, characterized in that: The performance evaluation results include F β1_score and F β2_score ; Among them, β1 and β2 are pre-set values ​​used to measure the importance of precision and recall; The positive and negative sample distinction index includes a first confidence threshold for distinguishing negative samples and a second confidence threshold for distinguishing positive samples; According to the performance evaluation results of the pre-trained target detection model under the confidence level of each inference result in each group of inference results, the positive and negative sample discrimination indicators are determined, including: Find F in a set of inference results corresponding to the maximum detection accuracy of the pre-trained target detection model. β1_score The confidence score of the inference result corresponding to the performance evaluation result with the largest value β1 ; Find F in a set of inference results corresponding to the maximum detection accuracy of the pre-trained target detection model. β2_score The confidence score of the inference result corresponding to the performance evaluation result with the largest value β2 ; Score β1 and score β2 The minimum value and the maximum value in are respectively determined as the first confidence threshold and the second confidence threshold.

8. The method according to claim 7, characterized in that Generating a pseudo label according to the positive and negative sample discrimination index and the group of inference results includes: Determine the inference results whose confidence is less than the first confidence threshold in the group of inference results, and generate a pseudo label indicating that the inference result is a negative sample for each determined inference result; An inference result having a confidence level greater than a second confidence threshold in the group of inference results is determined, and a pseudo label indicating that the inference result is a positive sample is generated for each determined inference result.

9. The method according to claim 1, characterized in that: The method further comprises: According to the generated pseudo labels, the pre-trained target detection model is trained to obtain a trained target detection model.

10. A pseudo-label generating device, characterized in that: The device includes: A training unit, used for pre-training a target detection model using a labeled first data set; A first reasoning unit is configured to use a pre-trained target detection model to perform reasoning on a second labeled data set under the NMS_IOU threshold for each of a plurality of preset NMS_IOU thresholds, and obtain a set of reasoning results corresponding to the NMS_IOU threshold; the NMS_IOU threshold is an overlap threshold used by the target detection model when executing the NMS algorithm during the reasoning process; A determination unit, configured to determine a positive and negative sample discrimination index and an optimal NMS_IOU threshold among the plurality of preset NMS_IOU thresholds according to a preset GP_IOU threshold, the annotation information of the second data set, and the plurality of groups of inference results corresponding to the plurality of preset NMS_IOU thresholds; the GP_IOU threshold is an overlap threshold set for the inference result and the annotation information corresponding to the inference result in the second data set; The second inference unit is used to use the pre-trained target detection model to infer a third data set that is homologous to the second data set under the optimal NMS_IOU threshold, obtain a set of inference results corresponding to the optimal NMS_IOU threshold, and generate a pseudo label according to the positive and negative sample discrimination index and the set of inference results.

Images