An active learning sample selection method for object detection
By using an active learning method to select representative samples and difficult-to-identify samples, the problem of insufficient sample size and excessive redundant information in object detection model training is solved, thereby improving the model's generalization ability and performance.
Patent Information
- Application Number
- CN202210897312.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-28
- Publication Date
- 2026-01-02
- Estimated Expiration
- 2042-07-28
AI Technical Summary
Existing technologies cannot effectively select representative samples and difficult-to-identify samples, resulting in a small number of samples and a large amount of redundant information in the labeled sample set during the training of the object detection model, which affects the model performance.
An active learning approach is adopted to use the object detection model to predict unlabeled sample data with high uncertainty. The object detection model is then retrained by adding labeled datasets. The retrained object detection model selects samples that are representative of the category and samples that are difficult to identify. The sample distribution is optimized by image feature similarity filtering and multi-instance method.
It improves the generalization ability of the target detection model, reduces the workload of data annotation, and enhances model performance.
Smart Images
Figure CN115393664B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of target detection model training, and in particular to a method for selecting active learning samples for target detection. BACKGROUND
[0002] The training of modern deep neural network models requires a large number of labeled data samples as support, but it is difficult to obtain high-quality labeled data sets, which requires a lot of manpower and resources. In addition, due to the different knowledge training of each person, it is very difficult and expensive to obtain standard labeling information. Sometimes, a large number of data sets are often filled with redundant information, which is of no benefit to the training of deep neural network models, and obtaining high-quality and widely representative samples has become a research hotspot in academia and industry.
[0003] Target detection is an algorithm model commonly used in the industry, which has a good application scenario and has a significant effect on improving industrial operation efficiency. However, the construction of the data set is also a difficulty encountered by the target detection model, and different enterprises need different data, and redundant information data sets often lead to a decrease in model performance. Therefore, there is an urgent need for a method for selecting samples for target detection models.
[0004] The prior art cannot use the method of active learning to select samples with class representation and difficult to identify, and the target detection model training process encounters problems such as small sample size and large amount of redundant information in the labeled sample set. SUMMARY
[0005] The present application is aimed at the deficiencies of the prior art and provides a method for selecting active learning samples for target detection. The method uses active learning to select sample data with high uncertainty for the target detection model in each round using the knowledge learned by the target detection model, and re-trains the target detection model by adding the labeled data set. The method selects samples with class representation and difficult to identify, so that the target detection model can learn the characteristics of these samples to enhance the generalization ability of the model, and better solves the problems encountered in the training process of the target detection model, such as small sample size and large amount of redundant information in the labeled sample set. The method is simple, has good use effect, and has certain application prospect.
[0006] The specific technical solution to achieve the object of the present application is: a method for selecting active learning samples for target detection, characterized by using the method of active learning to select sample data with high uncertainty for the target detection model in each round using the knowledge learned by the target detection model, and re-training the target detection model by adding the labeled data set. The method specifically includes the following steps:
[0007] Step 1: The target detection model predicts the uncertainty of the unlabeled data set
[0008] 1-1: Randomly select 1000 samples for annotation as the initial training set of the target detection model used, so that the target detection model learns the sample data features in the initial training set, and the trained target detection model is used as the selector of active learning.
[0009] 1-2: Use the trained target detection model (selector) to fix its model parameters, extract image example features from the labeled data set, and record the predicted classification results and confidence scores of the image example features. First, compare the similarity of the image internal example features, remove the example features with high similarity, and only keep the example features with low similarity, to improve the effective information ratio of a single image.
[0010] 1-3: On the basis of removing the internal redundant information in step 1-2, the image features of the labeled data set are clustered, and the similarity of the image features in different categories is calculated. At least 30% of the image features from each category are selected as image features for subsequent interpolation calculation. This method reduces the number of images required for interpolation and reduces the computational load.
[0011] 1-4: Use the trained target detection model (selector) to extract image features from the unlabeled data set, and perform interpolation one by one with the image features selected from the labeled data set in step 1-3 before outputting the prediction results. The interpolation ratio is 0.4 (the proportion of unlabeled data features), and then the interpolated features are input into the head part of the target detection model for result prediction output. The prediction result is subtracted from the confidence score of each category of the interpolated labeled image prediction result and the absolute value is taken. The calculation results of each category are accumulated to obtain the uncertainty of the unlabeled image. After the above operation is performed on the entire unlabeled data set, the obtained uncertainty is sorted to obtain the difficult-to-distinguish image samples as the image candidate set.
[0012] Step 2: Extract image effective information from the image candidate set
[0013] 2-1: The uncertainty of the image candidate set obtained from step 1 may have redundant information interference, and the similarity of the image example category features is high. In order to make the sample data information distribution of the next round of learning as extensive as possible, it is necessary to filter the similarity of the image features of the image candidate set. Each time, input all the example features and their classification confidence scores of an image, accumulate the confidence scores to obtain the image internal feature uncertainty, and then sort the image internal feature uncertainty to obtain the example candidate set of the image.
[0014] 2-2: Loop screening on the example candidate set of the image, selecting the feature with the highest uncertainty in each round to join the formal set, and deleting the example feature from the example candidate set, then calculating the distance between the remaining examples in the example set and the selected example in this round, and setting the similarity threshold to 60%, and deleting the example with higher similarity from the example candidate set. Repeat the above operation until the image example candidate set is empty, and obtain the final example set of the image.
[0015] 2-3: Repeat operation step 2-2 for each image. An unlabeled data set with redundant information in the image is obtained, and the uncertainty is updated and sorted due to the elimination of redundant information.
[0016] Step 3: Multi-example method to select minority class samples
[0017] 3-1: Count the number of examples and times of each class in the labeled image in each round, classify these examples according to
[0018] The number of times is divided into minority class and majority class.
[0019] 3-2: There may be an example imbalance phenomenon in the image candidate set, and some minority class samples appear less in the image, which makes the uncertainty method tend to majority class samples. In order to increase the proportion of minority class samples in the sample and make the sample class distribution conform to the normal distribution, a multi-example method is used to select images with minority class samples.
[0020] 3-3: The example with high uncertainty in the unlabeled image, i.e. difficult to identify example, can be regarded as minority class sample, and the image is selected into the minority class set.
[0021] 3-4: According to steps 3-2 and 3-3, the unlabeled data set is divided into minority class set and majority class set, and the labeled image is allocated according to the amount of the minority class set. Generally, the minority class set takes 60%, and the majority class set takes 40%. If the minority class set is empty or takes less than 60% of the labeled amount, it is obtained from the majority class set. In this way, the samples required for the next round of training of the target detection model are obtained.
[0022] The selector of step 1-2 refers to the model result obtained by the selected target detection model after initializing the model parameters in the last round and training on the new labeled data set, so that the model learns the features of the last round of labeled data set.
[0023] The method used in the step 1-2 of the internal image example similarity comparison is to express the example features as a vector form, and the Manhattan distance is used to calculate the similarity between each example feature, that is, the square of the difference of each component of the two vectors is calculated, and the results of all components are added and then the square root is taken to obtain the comparison value of the internal image example similarity.
[0024] The method used in the step 1-3 of the image similarity comparison is to delete the example features with high internal image similarity in the step 1-2, and only keep the example features with low similarity, and an image feature vector with a length equal to the number of categories is created, and the multi-classification confidence scores of all example features in the image are added to the corresponding image feature vector components one by one, and the final result is used as the image feature.
[0025] The method used in the step 2 of extracting the image candidate set is to perform similarity filtering on the data features in the unlabeled data set, and the image candidate set is obtained from the difficult-to-identify image samples in the step 1-4.
[0026] Compared with the prior art, the present application can effectively screen the unlabeled data set, select the difficult-to-identify samples and the unlabeled samples with wide representation, and label them, so as to enhance the generalization ability of the target detection model under the condition of the same number of labeled samples, select the samples with class representation and difficult-to-identify samples by using the active learning method, and further improve the generalization ability of the target detection model.
[0027] In the training process of the target detection model, the data labeling workload is large, the amount of labeled data information is low, and the model performance cannot be improved,
[0028] The method is simple, the use effect is good, and has certain application prospect. BRIEF DESCRIPTION OF DRAWINGS
[0029] Figure 1 The flowchart of the invention is shown in the figure;
[0030] Figure 2 The flowchart of the target detection model for calculating the uncertainty of the unlabeled image is shown in the figure;
[0031] Figure 3 The flowchart of the image candidate set redundancy information processing is shown in the figure;
[0032] Figure 4 The flowchart of the multi-example method for selecting a small number of class samples is shown in the figure. DETAILED DESCRIPTION
[0033] The application adopts an active learning method, selects sample data with high uncertainty for a target detection model in each round by using the knowledge learned by the target detection model, adds the labeled data set to retrain the target detection model, and specifically includes the following steps:
[0034] Step 1: The target detection model predicts the uncertainty of the unlabeled data set;
[0035] Step 2: Extract the image effective information of the image candidate set;
[0036] Step 3: A few-shot method is used to select the minority class sample.
[0037] Referring to Figure 1 , the application implements the construction of the image candidate set and removes redundant information by combining step 1 and step 2, and step 3 enhances the distribution of sample features, so that the newly labeled sample is representative, and specifically includes the following steps:
[0038] S400: The initial labeled data set is manually labeled by an expert, and is used as an initial learning data source of the model;
[0039] S410: The target detection model is trained according to the initial labeled data set, and an initial selector is obtained, which is used for subsequent sample selection;
[0040] S100-S120: Remove redundant information of the labeled data set, reduce the number of labeled data used for interpolation, so that the sample used for interpolation has high representativeness and can fully express the features;
[0041] S130: Feature interpolation is performed on the unlabeled data set and the selected labeled sample data set used for interpolation;
[0042] S140-S150: The target detection model classifies and judges the interpolated data, and obtains the image uncertainty;
[0043] S200-S250: According to the obtained image candidate set, the redundant information is removed to obtain a reordered image candidate set;
[0044] S300-S340: A few-shot method is used to assign the tendency of the next round of samples, and the proportion of minority classes and difficult-to-identify classes in images is as high as possible;
[0045] S350: The selected data is labeled by an expert;
[0046] S420: The newly labeled data is added to the original labeled sample set to generate a new labeled data set, and is added to the target detection model for training to train the selector of the next round.
[0047] The application trains a small amount of labeled data by using an existing target detection model, extracts image example features of the labeled data and unlabeled data by using the trained model, interpolates the features of the unlabeled data and the labeled data, performs prediction on the interpolated features, obtains the uncertainty of the unlabeled data set according to the classification result of the prediction output, and obtains a candidate set by sorting the uncertainty from high to low. Then, the image features in the candidate data set are calculated for similarity, the information redundancy of the candidate set is reduced, and representative samples are extracted. Then, a multi-example method is used to improve the priority of the minority class and enhance the distribution of the samples. The problems of large data labeling workload in the target detection model training process and low labeled data information that cannot improve the model performance are better solved.
[0048] In order to facilitate the understanding of the present application, the present application will be described in detail below in combination with the drawings and embodiments.
[0049] Embodiment 1
[0050] Referring to Figure 2 The target detection model trained in step 1 of the present application is used as an initial selector to select samples to be learned in the next round, and the specific steps are as follows:
[0051] S100: The initial sample set is used for the initial training of the target detection model, so that it has certain detection capability and can preliminarily extract image features and distinguish them;
[0052] S110-S120: First, the redundant information in each image of the labeled data set is removed, and then the redundant information between images is reduced to reduce the calculation amount for subsequent interpolation calculation;
[0053] S130: The labeled data set with the redundant information removed selects representative images in each category to be interpolated with unlabeled images;
[0054] S140-S160: The unlabeled images are interpolated with the labeled images, input into the target detection model, and if the detection result tends to be the classification result of the labeled images, it indicates that the information amount of the unlabeled image is low, and if the detection result is different from the labeled images, it represents that the information amount of the unlabeled image is good, and the uncertainty is preliminarily calculated and added to the image candidate set.
[0055] Referring to Figure 3 The redundant information in the image candidate set is removed in step 2 of the present application, and the specific steps are as follows:
[0056] S200-S210: One image is selected from the image candidate set, and all example features of the image are extracted to obtain an example candidate set;
[0057] S220-S230: Similarity comparison is performed on examples in the example candidate set, and examples with high similarity are filtered out to obtain an image with internal redundant information removed;
[0058] S240-S250: The images in the image candidate set are subjected to example similarity removal to obtain an image candidate set with redundant information removed.
[0059] Referring to Figure 4 The step 3 of the present application uses a multi-example method to assign weights to the image candidate set, and increases the number of labels of the minority class as much as possible. The specific steps are as follows:
[0060] S300-310: The classification information in the currently labeled data set is counted, and the number of examples of different classes is counted.
[0061] The minority class and the majority class are divided in proportion, the first 60% of the classification number is the majority class, and the last 40% of the classification number is
[0062] The minority class;
[0063] S320: The multi-example method is used to judge the minority class in the image candidate set. As long as one example of the minority class is included, the weight is increased, and the more the minority class examples, the higher the weight.
[0064] S330: The image sample that is difficult to be distinguished by the current target detection model is regarded as the minority class, and the weight is the highest.
[0065] S340-350: The image candidate set is sorted according to the weight assigned by the multi-example method, and the minority class accounts for 60% in the next round of labeling data. If the number of minority classes does not meet the requirement, the images are selected for labeling according to the uncertainty.
[0066] The above is only a further description of the present application, and is not used to limit the patent. Any equivalent implementation of the present application shall be included in the scope of the claims of the present patent.
Claims
1. An active learning sample selection method for object detection, characterized in that The method of active learning is used to select sample data with high uncertainty for the target detection model in each round, and the labeled data set is added to retrain the target detection model, which specifically includes the following steps. Step 1: The target detection model predicts the uncertainty of the unlabeled data set 1-1: Randomly select 1000 samples from the unlabeled data set for labeling as the initial training set for the target detection model, so that the target detection model learns the features of the sample data in the initial training set, and the trained target detection model is used as the selector of active learning; 1-2: Using the trained target detection model, fix its model parameters, extract the image feature of the labeled data set, record the predicted classification result and confidence score of the image feature, and compare the similarity of the image internal features, remove the features with high similarity, and only keep the features with low similarity, so as to improve the effective information ratio of single image; 1-3: On the basis of removing the internal redundant information of the image in step 1-2, the image features of the labeled data set are clustered, the similarity of the image features in different categories is calculated, and at least 30% of the image features in each category are selected as the image features used for subsequent interpolation calculation; 1-4: Using the trained target detection model, the image features of the unlabeled data set are extracted, and before the prediction result is output, the image features selected in step 1-3 are interpolated one by one, the interpolation ratio is 40% of the unlabeled data features, the interpolated features are input into the head part of the target detection model for result prediction output, the predicted result is subtracted from the confidence score of each classification of the interpolated labeled image, and the absolute value is taken, the calculation results of each classification are accumulated, and the uncertainty of the unlabeled image is obtained, the above operation is performed on the entire unlabeled data set, and the obtained uncertainty is sorted to obtain the difficult-to-distinguish image samples as the image candidate set; Step 2: Extract image effective information of image candidate set 2-1: Similarity filtering is performed on the image features of the image candidate set, all example features and their classification confidence scores of an image are input each time, the confidence scores are accumulated to obtain the feature uncertainty within the image, and the feature uncertainty within the image is sorted to obtain the example candidate set of the image; 2-2: The example candidate set of the image is selected in a loop, the feature with the highest uncertainty is selected in each round and added to the formal set, and the example feature is deleted from the example candidate set, then the distance between the remaining examples in the example set and the selected example in this round is calculated, and the similarity threshold is set to 60%, the examples with higher similarity are deleted from the example candidate set, and the above operation is repeated until the image example candidate set is empty, and the final example set of the image is obtained; 2-3: Repeat operation step 2-2 for each image to obtain the unlabeled data set with redundant information removed from the image, update the uncertainty of each image and sort it; Step 3: Select minority class samples by multi-example method 3-1: Count the number of examples of each class in the labeled images of this round, and classify these examples according to the number of times, and divide them into minority classes and majority classes according to the number of times; 3-2: Select images with minority class samples using the multi-instance method; 3-3: Consider the difficult-to-identify examples as minority class samples, and select their images into the minority class set; 3-4: Divide the unlabeled data set into minority class set and majority class set according to steps 3-2 and 3-3, The quota for labeling images in this round is inclined to the minority class set, that is, the minority class set takes 60%, and the majority class set takes 40%. If the minority class set is empty or does not take 60% of the labeling quota, it is all taken from the majority class set. In this way, the samples required for the next round of training of the target detection model are obtained. 2.The active learning sample selection method for object detection according to claim 1, characterized in that The selector refers to the selected target detection model, which initializes the model parameters in the last round and trains on the new labeled data set to obtain a model result that learns the features of the last round of labeled data set. 3.The active learning sample selection method for object detection according to claim 1, characterized in that The similarity comparison of the example features in the image in step 1-2 is to express the example features as a vector form, and the similarity between each example feature is calculated by using Manhattan distance. Subtract each component of the two vectors and square the value, and then take the square root of the cumulative result of all components to obtain the comparison value of the internal image example similarity. 4.The method for active learning sample selection for object detection according to claim 1, wherein The similarity calculation of the image features in step 1-3 is to delete the example features with high similarity in the image in step 1-2, and only keep the example features with low similarity. The image feature is created as an image feature vector with a length equal to the number of classes. The multi-class confidence scores of all example features in the image are added one by one to the corresponding image feature vector components, and the final result is used as the image feature.
5. The method of claim 1, wherein The step 2 extracts the image effective information of the image candidate set, which is a similarity filtering of the data features in the unlabeled data set. The image candidate set is obtained from step 1-4.