A semi-supervised learning active method based on image key data extraction

Abstract

The application discloses a semi-supervised learning active method based on image key data extraction, first, a small part of samples are selected randomly from a no-label sample pool for labeling to obtain an initial labeled sample set and a no-label sample set, the labeled set is used for supervised model training, and the no-label sample set with the maximum confidence greater than or equal to a threshold value is used for unsupervised model training to obtain a model after first iteration; then, from the no-label sample with the maximum confidence less than the threshold value, an active learning algorithm based on uncertainty and diversity is used to query the sample with the most abundant information for labeling; next, the labeled sample set and the no-label sample set are updated according to the selected sample; the above process is repeated until a preset model precision is reached or a labeling budget is exhausted. The application improves the classification performance and can provide more reliable information for a sample selection strategy.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology, specifically relating to a semi-supervised learning active method based on key image data extraction. Background Technology

[0002] In recent years, with the emergence of large-scale labeled data, supervised deep learning has developed rapidly, especially in the field of image classification, where it has achieved great success. As deep learning has gradually matured, image classification has been applied to various fields, such as facial recognition in security, traffic scene recognition in transportation, and medical image classification in medicine, which has greatly facilitated people's daily lives.

[0003] While supervised deep learning has achieved significant results in image classification, its success relies heavily on large amounts of labeled data. However, in real-world scenarios, labeling large datasets is extremely expensive and time-consuming. For example, in the biomedical field, image annotation can only be performed by experts with relevant domain knowledge, and the shortage of such talent further exacerbates the cost of labeled data. Due to the high cost of annotation and the limitations of certain domains, sufficient labeled data is unavailable, preventing current mainstream models from fully realizing their performance advantages. Furthermore, labeling a large number of samples is a repetitive and tedious task, prone to errors that negatively impact subsequent learning tasks. Therefore, reducing the number of labeled samples and saving on annotation costs while maintaining model accuracy is crucial, and this holds significant practical importance in image classification as well.

[0004] Active learning aims to train a model with the best possible performance using as few labeled samples as possible. Active learning is a iterative process of selecting information-rich samples from the unlabeled sample set and assigning them to expert labelers until the desired model accuracy is achieved or the labeling budget is exhausted. Its core lies in the design of the sample selection strategy, which uses specific sampling methods to select a small number of high-quality samples for labeling. By using fewer samples without affecting model accuracy, it significantly saves on labeling costs. Therefore, active learning has very high economic value and promising application prospects. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a semi-supervised active learning method based on key image data extraction. First, a small subset of samples is randomly selected from an unlabeled sample pool for labeling, resulting in an initial labeled and unlabeled sample set. Supervised model training is performed using the labeled set, followed by unsupervised model training using the unlabeled sample set with a maximum confidence score greater than or equal to a threshold, yielding the model after the first iteration. Then, from the unlabeled samples with a maximum confidence score less than the threshold, an active learning algorithm based on uncertainty and diversity is used to query and label the samples with the richest information. Next, the labeled and unlabeled sample sets are updated based on the selected samples. This process is repeated until a preset model accuracy is achieved or the labeling budget is exhausted. This invention improves classification performance and provides more reliable information for sample selection strategies.

[0006] The technical solution adopted by this invention to solve its technical problem includes the following steps:

[0007] Step 1: Randomly select a portion of samples from the unlabeled image sample pool for labeling to obtain the initial labeled sample set L0 and the unlabeled sample set U;

[0008] Step 2: Perform weak enhancement on labeled samples, and weak and strong enhancement on unlabeled samples respectively;

[0009] Step 3: Using labeled samples, supervise the optimization training of the active learning model parameters using the cross-entropy loss function;

[0010] Step 4: For unlabeled samples, if the maximum confidence level predicted by the current model is greater than the preset threshold, the category corresponding to that confidence level is taken as the true label, and the consistency loss is used for unsupervised model training.

[0011] Step 5: Using an active learning algorithm based on uncertainty and diversity, select the top b samples with the highest information content scores from the unlabeled samples whose maximum confidence predicted by the current model is less than a preset threshold, and hand them over to sample annotation experts for annotation.

[0012] Step 6: Add the unlabeled samples labeled in Step 5 to the labeled sample set L0, and remove the labeled unlabeled samples from the unlabeled sample set U.

[0013] Step 7: Repeat steps 1 to 6 above until the preset model accuracy is reached or the sample annotation budget is exhausted.

[0014] Preferably, the weak enhancement operation includes random rotation, random cropping, and random changes to the brightness and contrast of the image.

[0015] Preferably, the enhanced brightness includes brightness transformation, random rotation, exposure, and tone separation operations.

[0016] Preferably, the cross-entropy loss function in step 3 is as follows:

[0017]

[0018] Where B represents the sample batch size, p p Labeled sample x b The one-hot encoding of the real label y, x b It is the b-th sample, y is its label, weak(x) b ) is x b The result after weak enhancement, p m (y∣weak(x b )) is the model's prediction for the weakly enhanced sample in class y.

[0019] Preferably, step 4 is as follows:

[0020] Step 4-1: Based on the version obtained after weak augmentation of each unlabeled sample using the current model, predict the probability distribution of its category;

[0021] Step 4-2: If the maximum confidence score of the prediction in the weakly enhanced version's prediction probability distribution is greater than a preset threshold, then the category corresponding to that value is used as the pseudo-label for the current unlabeled sample, and then unsupervised training is performed using the following formula:

[0022]

[0023] Where `strong()` represents a strong augmentation operation on unlabeled samples, and `τ` is a hyperparameter threshold. It is the one-hot encoding of the pseudo tag, u b These are unlabeled samples, B represents the batch size, and p m (y∣strong(u b )) represents the distribution of sample x categories predicted by the model.

[0024] Preferably, step 5 is performed as follows:

[0025] Step 5-1: Extract feature representations for all samples using the current model;

[0026] Step 5-2: In the feature space, for each unlabeled sample, calculate its K nearest labeled neighbors;

[0027] Step 5-3: Use the current model to predict the probability distribution of the current unlabeled sample and its K nearest neighbors;

[0028] Step 5-4: Calculate the uncertainty score of the sample using the following formula to measure the dissimilarity of the class distributions:

[0029]

[0030] in Representative through model predict The probability of belonging to category c, where C represents the total number of all categories, and Var represents the function for calculating variance;

[0031] Step 5-5: Calculate the diversity score of the current candidate sample in the feature space, using the following formula:

[0032]

[0033] Where distance (1) In the feature space, density represents the distance between the current unlabeled sample and the nearest labeled sample, and density represents the density of unlabeled samples in the region where the current candidate sample is located. Specifically, the average distance between the current candidate sample and the k nearest unlabeled samples is calculated and used as the density score of the sample. The specific formula is as follows:

[0034]

[0035] In the formula d j This represents the distance between the current candidate sample and its nearest j-th sample;

[0036] Steps 5-6: Sort the uncertainty score and diversity score from smallest to largest, and then add the position indices of each candidate sample in the two ordered sequences to obtain the final information content metric score.

[0037] Steps 5-7: Sort the samples from largest to smallest based on their information content metric scores, and select the top b samples with the highest information content metric scores.

[0038] Preferably, in step 7:

[0039] If the labeling budget is exhausted or the preset model accuracy has been reached, active learning stops and the optimal model is output; otherwise, the samples selected in step 5 are added to the labeled sample set and these samples are deleted from the unlabeled sample set to complete the update of the labeled and unlabeled sample sets, and then the next round of training is carried out.

[0040] The beneficial effects of this invention are as follows:

[0041] (1) This invention combines semi-supervised learning and active learning strategies, enabling the model to learn stronger feature representations from labeled and unlabeled samples, thereby improving classification performance and providing more reliable information for sample selection strategies. The high-uncertainty samples selected by active learning help the semi-supervised model eliminate difficult samples that are hard to predict accurately, thus reducing the uncertainty of the model's prediction and facilitating the rapid learning of the semi-supervised model.

[0042] (2) The present invention designs a new sample selection strategy that enables the final selected sample to take into account uncertainty and diversity. Compared with existing active learning algorithms, the high-quality samples selected by this method are mostly concentrated near the decision boundary with relatively high information content. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] This invention proposes a semi-supervised active learning method based on key data extraction, overcoming the high cost of labeled samples in existing active learning methods. This invention combines a semi-supervised framework with active learning, which not only fully utilizes the information contained in unlabeled samples but also proposes a novel sample selection strategy that balances uncertainty and diversity in the final selected samples.

[0046] like Figure 1 As shown, to achieve the above objectives, the specific steps of this invention are as follows:

[0047] (1) Randomly select a small portion of samples from the unlabeled sample pool and hand them over to sample labeling experts to label them, thus obtaining the initial labeled sample set L0 and the unlabeled sample set U.

[0048] (2) Supervised model training is performed using a labeled set, and then unsupervised model training is performed using an unlabeled sample set with a maximum confidence level greater than or equal to the threshold τ to obtain the model after the first iteration.

[0049] (3) From the unlabeled samples with the maximum confidence level less than the threshold τ, use an active learning algorithm based on uncertainty and diversity to query the samples with the richest information and label them.

[0050] (4) Update the labeled and unlabeled sample sets based on the selected samples.

[0051] (5) Repeat the above process until the preset model accuracy is reached or the annotation budget is exhausted.

[0052] Step (3) combines the uncertainty score with the diversity score as an information measure of the sample. The specific steps include:

[0053] 3.1 Based on the current model, extract feature representations for all samples (labeled and unlabeled samples).

[0054] 3.2 For each unlabeled sample, use a distance metric function in the feature space to calculate its K nearest neighbor samples.

[0055] 3.3 Use the current model to predict the probability distribution of the current candidate sample and the class of its K nearest neighbors.

[0056] 3.4 Calculate the variance of the K+1 probability distributions to obtain the uncertainty score of the sample.

[0057] 3.5 In the feature space, distance and density are considered to calculate the diversity score.

[0058] 3.6 Sort the samples according to the two scores respectively, and add up the corresponding positions of each sample in the two ascending sequences to get the final score for measuring the information content of the samples. Specific implementation examples:

[0060] Step 1: Randomly select a small portion of samples from the unlabeled sample set and hand them over to sample annotation experts for annotation, to obtain the initial labeled sample set L0 and the unlabeled sample set U.

[0061] Step 2: Perform weak enhancement (including random rotation, random cropping, and random changes to image brightness and contrast) on labeled samples, and perform weak and strong enhancement (including brightness transformation, random rotation, exposure adjustment, and tone separation) on unlabeled samples respectively.

[0062] Step 3: Perform supervised training using the enhanced labeled set; for the weakly enhanced unlabeled samples, if the maximum confidence predicted by the current model is greater than or equal to the threshold τ, then perform unsupervised training to obtain the initial semi-supervised model. In the specific implementation, ResNet-18 can be selected as the backbone to learn the feature information of the samples, thereby providing more reliable output information for the design of subsequent sample selection strategies.

[0063] Step 4: From the unlabeled samples with a maximum confidence level less than the threshold τ, use the active learning algorithm to query the samples with the richest information and label them.

[0064] 4.1 Using the current model, extract feature representations for all samples (labeled and unlabeled samples).

[0065] 4.2 For each unlabeled sample, use a distance metric function in the feature space to calculate its K nearest neighbor samples {X1, ..., X...} K}

[0066] In practice, either the Euclidean distance formula or the cosine distance formula can be used. The Euclidean distance formula for calculating the distance between labeled and unlabeled samples is shown below:

[0067]

[0068] 4.3 Using the current model to infer the probability distribution of the current candidate sample and its K nearest neighbors' classes, the resulting class probability distribution of the K+1 samples can be expressed as:

[0069]

[0070] 4.4 The variance function is used to calculate the variability of the K+1 distributions. The final variance value is used as the uncertainty score of the sample. The formula for measuring the variability of the class distributions using the variance function is shown below:

[0071]

[0072] 4.5 Considering the redundancy between samples, the diversity score of the current candidate samples is calculated. Simultaneously, to prevent the selection of outliers, the diversity score is calculated by considering both the distance between samples and their distribution density in the feature space. The diversity score is expressed by the following formula:

[0073]

[0074] Where the density satisfies the following formula:

[0075]

[0076] Where distance (1) The larger the value, the farther the current candidate sample is from the labeled sample, and the higher the diversity. Density represents the density of the current candidate sample. It can be seen that the farther away from the labeled sample, the higher the density, and the higher the diversity score of the current candidate sample.

[0077] 4.6 Sort the candidate samples according to their two scores: uncertainty score and diversity score. Sum the corresponding positions of each sample in the two ascending sequences to get the final measure of the information content score of the sample.

[0078] Step 5: Select the top b samples with the highest information content scores from the unlabeled sample set, and use L... query This indicates that the sample will then be handed over to sample labeling experts to assign a label.

[0079] Step 6: Then update the labeled and unlabeled sample sets to satisfy the following formula:

[0080] The update for the labeled sample set is as follows:

[0081] L t =L t-1 ∪L query

[0082] The update for the unlabeled sample set is as follows:

[0083] U = U / L query

[0084] Step 7: Determine if the annotation budget has been exhausted or if the model accuracy is met.

[0085] If the above conditions are met, the active learning cycle ends.

[0086] If the above conditions are not met, repeat the above steps.

Claims

1. A semi-supervised learning active method based on image key data extraction, characterized in that, The method comprises the following steps: Step 1: randomly select part of samples from the image unlabeled sample pool for labeling to obtain an initial labeled sample set L0 and an unlabeled sample set U; Step 2: perform weak enhancement operation on the labeled samples, and perform weak enhancement and strong enhancement operations on the unlabeled samples respectively; Step 3: using the labeled samples, the active learning model parameters are supervisedly optimized and trained by using a cross-entropy loss function; Step 4: for the unlabeled samples, if the maximum confidence predicted by the current model is greater than a preset threshold, the class corresponding to the confidence is taken as a real label, and unsupervised model training is performed by using a consistency loss; Step 5: using an active learning algorithm based on uncertainty and diversity, the first b samples with the highest information quantity score are selected from the unlabeled samples whose maximum confidence predicted by the current model is less than the preset threshold, and the samples are given to a sample labeling expert for labeling; Step 6: the unlabeled samples labeled in step 5 are added to the labeled sample set L0, and the labeled unlabeled samples are removed from the unlabeled sample set U; Step 7: repeat steps 1 to 6 until a preset model accuracy is reached or a sample labeling budget is exhausted; In the step 4, the following steps are performed: Step 4-1: according to the current model, the class probability distribution of each unlabeled sample is predicted after weak enhancement; Step 4-2: if the maximum confidence in the predicted probability distribution of the weak enhancement version is greater than a preset threshold, the class corresponding to the confidence is taken as the pseudo label of the current unlabeled sample, and then unsupervised training is performed by using the following formula: where strong() represents strong augmentation operation on the unlabeled samples, is a hyper-parameter threshold, is one-hot encoding of pseudo labels, is an unlabeled sample, B denotes the sample batch size, denotes the model predicted sample x class distribution. 2.The semi-supervised learning active method based on image key data extraction of claim 1, wherein, The weak enhancement operation comprises random rotation, random cropping, random change of image brightness and contrast. 3.The semi-supervised learning active method based on image key data extraction of claim 1, wherein, The strong enhancement comprises brightness transformation, random rotation, exposure and hue separation operation. 4.The semi-supervised learning active method based on image key data extraction of claim 1, wherein, The cross-entropy loss function of step 3 is as follows: wherein B denotes the sample batch size, is a labeled sample with one-hot encoding of its true label y, is the b th sample, y is its label, weak( ) is the result of weak augmentation, is the model’s prediction for the weak augmented sample on the y th class.

5. The semi-supervised learning active method based on image key data extraction according to claim 4, characterized in that, The specific operation of step 5 is as follows: Step 5-1: using the current model, the feature representation of all samples is extracted; Step 5-2: in the feature space, for each unlabeled sample, the nearest K labeled neighbor samples are calculated; Step 5-3: using the current model, the probability distribution of the current unlabeled sample and its nearest K samples is predicted; Step 5-4: the difference of the class distribution is measured by using the following formula to calculate the uncertainty score of the sample: wherein representing passing through a model predicting the probability of belonging to a class c, C represents the total number of all classes, and Var represents a function for calculating variance; Step 5-5: in the feature space, the diversity score of the current candidate sample is calculated, and the specific formula is as follows: wherein represents the distance between the current unlabeled sample and the nearest labeled sample in the feature space, and density represents the density of the unlabeled samples in the region where the current candidate sample is located; specifically, the average distance between the current candidate sample and the nearest k unlabeled samples is calculated as the density score density of the sample, and the specific formula is represented as: wherein denotes the distance between the current candidate sample and its nearest jth sample; Step 5-6: the uncertainty score and the diversity score are sorted from small to large respectively, and then the bit indexes of each candidate sample in the two ordered sequences are added to obtain the final information quantity measurement score; Step 5-7: according to the information quantity measurement score, the first b samples with the highest information quantity measurement score are selected from large to small.

6. The semi-supervised learning active method based on image key data extraction according to claim 5, characterized in that, In the step 7: If the labeling budget is exhausted or the preset model accuracy is reached, the active learning is stopped, and the optimal model is output; otherwise, the samples selected in step 5 are added to the labeled sample set, and the samples are deleted from the unlabeled sample set, the labeled sample set and the unlabeled sample set are updated, and then the next round of training is performed.

Images