Dual-space adaptive sampling active learning labeling method, medium and device

Abstract

This invention provides a dual-space adaptive sampling active learning annotation method, medium, and device, belonging to the field of computer vision. The method includes: extracting feature layer vectors and prediction probability vectors from unlabeled dataset samples; calculating edge uncertainty metrics; when the selected sample set is empty, using the sample with the largest edge uncertainty metric as the initial anchor point; calculating the probability space distance and feature space distance based on the initial anchor point, taking the minimum value as the probability space diversity component and feature space diversity component, and weighted fusing them to obtain the dual-space diversity contribution; constructing a nonlinear adaptive comprehensive score based on the edge uncertainty metric and the dual-space diversity contribution; calculating the sample selection probability and performing sampling without duplication; adding samples and repeating the sampling steps when the number of samples in the selected sample set does not reach a preset number; submitting for expert annotation after reaching the preset number, adding the sample to the labeled training set to drive model retraining.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a dual-space adaptive sampling active learning annotation method, medium, and device. Background Technology

[0002] In the field of computer vision, the performance of image recognition models is highly dependent on the support of large-scale, high-quality labeled datasets. As real-world application scenarios continue to evolve, deployed image recognition models need to periodically introduce new categories of samples or adapt to data distribution shifts, thus creating a continuous demand for manual annotation. Traditional fully supervised annotation methods involve manually annotating all newly acquired image data, and the annotation cost increases linearly or even superlinearly with the data size, severely limiting the iterative update efficiency of image recognition models.

[0003] Active learning, by constructing an intelligent sample query mechanism, maximizes the performance gain of image recognition models under the constraint of limited annotation budget. It is the core technical path to reduce the cost of image annotation. In the active learning framework based on data pool, the model maintains an unlabeled image data pool and an labeled training set. In each iteration, the query strategy selects several image samples with the most annotation value from the unlabeled image data pool, submits them to experts for annotation, adds them to the labeled training set, and drives the model to retrain. This process is repeated until the upper limit of the annotation budget is reached.

[0004] Existing active learning query strategies can be categorized as follows: Uncertainty-based sampling strategies measure sample information content by the predicted probability distribution output by the image classification model, selecting image samples with the least uncertain model predictions. Typical methods include margin sampling, entropy sampling, and least confidence sampling. These methods can effectively identify high-information images near the decision boundary, but in batch query scenarios, they tend to select redundant samples with highly similar image content, and rely entirely on probability prediction while ignoring the diversity of semantic features in the images. Diversity-based sampling strategies aim to maximize the coverage of the image feature space. Typical methods, such as the core set, find the sample subset with the smallest coverage radius in the feature layer embedding space. This can effectively ensure the semantic diversity of the image, but it completely ignores the actual contribution of each image sample to the refinement of the current model boundary, resulting in low efficiency in the use of the annotation budget. The committee-based query strategy involves maintaining multiple image classification models to form a committee, using the prediction discrepancy among members to measure the amount of sample information. The computational cost increases linearly with the size of the committee, resulting in high engineering deployment costs. The strategy based on expected model changes selects image samples that produce the largest gradient changes in model parameters to maximize the benefits of each round of annotation budget for updating model parameters. This usually requires additional forward or backward propagation and has high computational complexity.

[0005] The above query strategies have the following drawbacks: Sampling strategies based solely on uncertainty suffer from severe image sample redundancy in batch query scenarios, resulting in insufficient coverage of the image feature space; sampling strategies based solely on diversity aim for uniform coverage of the image feature space, completely ignoring the prediction uncertainty of image samples, leading to low efficiency in annotation budget utilization; existing hybrid strategies linearly combine uncertainty and diversity with fixed weights, failing to adapt adaptively to the dynamic construction process of query batches; diversity measurement is limited to a single feature space, failing to simultaneously guarantee sample dispersion in both the prediction probability dimension and the image semantic feature dimension; existing batch query strategies lack an effective cold start mechanism in the initial stage, making the quality of the first anchor sample uncontrollable and affecting the quality of the entire batch query results.

[0006] Chinese invention patent application document with publication number CN116720570A discloses an active learning method, device and storage medium based on data uncertainty and diversity. The method alleviates the redundancy problem caused by uncertainty by using sample diversity. However, the method lacks a unified sample evaluation mechanism for uncertainty and diversity. The two types of evaluation are carried out independently, making it difficult to simultaneously constrain the sample redundancy problem in the prediction space and feature space.

[0007] Chinese invention patent application document with publication number CN113590830A discloses an active learning classification method based on uncertainty and similarity measurement, which integrates uncertainty and similarity through weighting. However, this method only uses linear weighting and lacks adaptive ability. Summary of the Invention

[0008] To address the problems in the prior art, this invention proposes a dual-space adaptive sampling active learning annotation method, medium, and device. The method constructs a dual-space diversity measure in probability space and feature space, and combines it with an uncertainty measure to construct a nonlinear adaptive comprehensive scoring function. This allows samples to be sufficiently dispersed in both the prediction mode and image content dimensions, driving the image recognition model to obtain more balanced semantic coverage under the same annotation budget, thereby improving annotation efficiency.

[0009] The technical solution of the present invention is as follows: In a first aspect, this invention proposes a dual-space adaptive sampling active learning annotation method, comprising the following steps: Perform feature extraction and classification mapping on samples in the unlabeled dataset to obtain the corresponding feature layer vector and prediction probability vector; Calculate the marginal uncertainty measure for each sample based on the predicted probability vector; When the selected sample set is empty, the sample with the largest marginal uncertainty measure is added to the selected sample set as the initial uncertainty anchor point; Based on the initial anchor point of uncertainty, calculate the probability space distance and feature space distance between each sample in the unselected sample set and each sample in the selected sample set, and take the minimum value as the probability space diversity component and the feature space diversity component. Perform the sampling steps: The diversity components in the probability space and the diversity components in the feature space are normalized and then weighted and fused to obtain the diversity contribution of the two spaces. A nonlinear adaptive comprehensive score is constructed based on the marginal uncertainty measure and the contribution of dual-space diversity. The sample selection probability is calculated through the nonlinear adaptive comprehensive score. Based on the sample selection probability, non-repeating sampling is performed and the samples are added to the selected sample set. If the number of samples in the selected sample set does not reach the preset number, add the unselected samples one by one and repeat the sampling step; Once the preset number is reached, submit the code to experts for annotation, and add it to the already annotated training set to drive model retraining.

[0010] Furthermore, when the selected sample set is empty, the probability space diversity component is initialized based on the probability space distance of the uncertain initial anchor point; the feature space diversity component is initialized based on the feature space distance of the uncertain initial anchor point.

[0011] Furthermore, the marginal uncertainty metric is the maximum value of 1 minus the difference between the predicted probability of the largest category and the predicted probability of the second largest category.

[0012] Furthermore, the probability space distance The L1 norm, based on the predicted probability vector, is used as a metric, expressed by the formula: ; in, Indicates the first The sample was predicted as the first... A probability vector for each category; Indicates the first The sample was predicted as the first... A probability vector for each category; This is for absolute value operations; The feature space distance The cosine distance, based on the feature layer vector representation, is used as the metric, expressed by the formula: ; in, Indicates the first Feature layer vectors of each sample; Indicates the first Feature layer vectors of each sample; The norm of a vector.

[0013] Furthermore, the contribution of dual-space diversity is obtained by weighting and fusing the probability space diversity component and the feature space diversity component using a fusion coefficient, wherein the fusion coefficient is... .

[0014] Furthermore, the nonlinear adaptive comprehensive scoring By measuring edge uncertainty and the contribution of dual-space diversity The result is obtained by summing after performing a nonlinear transformation, and can be expressed as a formula: ; in, This is a non-linear adjustment parameter.

[0015] Furthermore, the sample selection probability is expressed by the formula: ; in, For the first The sample selection probability of each sample; U is the unlabeled dataset; For the first Nonlinear adaptive comprehensive score for each sample.

[0016] Furthermore, feature extraction is performed using convolutional neural networks or attention-based neural networks.

[0017] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods described above.

[0018] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the methods described above.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention constructs a dual-space diversity measure in probability space and feature space, and combines it with uncertainty measure for unified modeling. During the sample selection process, it simultaneously constrains the differences in prediction results and semantic features, effectively avoiding the sample redundancy problem caused by single-space sampling. By introducing a nonlinear adaptive comprehensive scoring function, the uncertainty and diversity components are uniformly adjusted, thereby enhancing the scoring of high-value samples and improving the discriminative power and labeling efficiency of sample selection. The entire method only nonlinearly adjusts the parameters. and weighted fusion coefficient With two hyperparameters, the tuning cost is significantly lower than that of a multi-parameter hybrid strategy; Meanwhile, the cold start mechanism with uncertain initial anchor points ensures that the first selected sample has a high information content and provides a stable reference benchmark for subsequent sample selection, thereby improving the consistency and reliability of batch sampling results. Attached Figure Description

[0020] Figure 1 This is a flowchart of a dual-space adaptive sampling active learning annotation method; Figure 2 Flowchart for initializing anchor points for cold start. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0022] Example 1: This embodiment provides a dual-space adaptive sampling active learning annotation method, such as... Figure 1 As shown, it includes the following steps: S1. Perform forward inference of the feature extraction network on all images in the unlabeled image data pool U. The feature extraction network includes a feature extraction module. and classification mapping module Feature extraction module Extracting the feature layer vector of the image Classification mapping module Map image features to a category space and output the predicted probability vector for each category. Expressed as a formula: ; ; Where softmax(·) is the softmax activation function; express 3D feature vector; express 3D class probability vector; The feature extraction network includes a convolutional neural network and an attention-based neural network; S2, Calculate the marginal uncertainty measure Marginal uncertainty measure is the category with the highest predicted probability. and the second largest category The difference in probabilities with respect to the maximum value of 1 can be expressed by the formula: ; ; ; Where arg max represents taking the class with the highest predicted probability; For image Predicted as the first A probability vector for each category; The value ranges from [0,1]. A larger value indicates a smaller prediction confidence difference between the optimal and suboptimal classes, meaning a higher difficulty in image classification and a greater amount of information for the current model. Compared to entropy sampling and minimum confidence, marginal uncertainty creates multiple local extrema in the probability space, promoting sample diversity in the uncertainty dimension. It does not converge the query batch to the region of maximum global entropy, thus synergizing with subsequent diversity mechanisms. S3, Calculate the distance in the probability space. and feature space distance The probability space distance The feature space distance is measured using the L1 norm based on the predicted probability vector. The cosine distance, based on the feature layer vector representation, is used as the metric, expressed by the formula: ; ; in, Indicates the first The image was predicted as the first... A probability vector for each category; Indicates the first The image was predicted as the first... A probability vector for each category; Indicates the first Feature layer vectors of an image; Indicates the first Feature layer vectors of an image; The norm of a vector; S4. Determine if the selected sample set Q is empty. If the selected sample set Q is empty, use the marginal uncertainty measure. The largest image serves as the initial anchor point for uncertainty. Add to the selected sample set Q, and use the probability space distance and feature space distance Initialize the diversity components of the probability space and feature space diversity components ; The cold start anchor point initialization process is as follows: Figure 2 As shown, this strategy ensures that the first selected sample has the highest information content. Subsequent sample selection is iteratively updated based on the distance to the set of selected samples, so that the sample moves away from the selected samples in both the probability space and the feature space, spreading in the direction with the greatest difference in the two dimensions. This significantly improves the consistency of anchor quality compared to random initialization. S5. Add images one by one to the selected sample set Q, update the minimum probability space distance and feature space distance between the unselected sample set and the selected sample set Q, and perform max-min normalization to obtain the probability space diversity components. and feature space diversity components ; The unselected sample set consists of the remaining images in the unlabeled image data pool U; S6. Diversity components in probability space and feature space diversity components Weighted fusion is performed to obtain the contribution of dual-space diversity. Expressed as a formula: ; in, The weighted fusion coefficient; S7. Measuring edge uncertainty and the contribution of dual-space diversity Obtain nonlinear adaptive comprehensive score Expressed as a formula: ; in, It is a non-linear adjustment parameter; By applying the same nonlinear transformation to both the uncertainty component and the diversity component, high-value components are amplified while low-value components are suppressed, thereby achieving adaptive adjustment of the degree of influence of different components without the need to manually specify a fixed weight ratio. S8. Incorporate nonlinear adaptive comprehensive scoring Normalization is performed to obtain the selection probability of each image sample. Expressed as a formula: ; in, For the first Nonlinear adaptive comprehensive scoring of images; Based on the selection probability The image samples are randomly sampled, and the selected samples are removed from the unlabeled image data pool U after each selection to achieve non-repeating sampling based on the sample selection probability. The selected samples are then added to the selected sample set Q. S9. Determine whether the number of images in the selected sample set Q has reached the preset number N. If it has not reached the preset number N, return to step S5 and loop. After reaching the preset number, submit expert annotation and add it to the annotated training set L to drive model retraining.

[0023] Example 2: This embodiment proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in Embodiment 1 of the present invention.

[0024] Example 3: This embodiment proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method described in Embodiment 1 of this invention.

[0025] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure made using the contents of the present invention specification and drawings, or directly or indirectly applied to other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A dual-space adaptive sampling active learning annotation method, characterized in that, Includes the following steps: Perform feature extraction and classification mapping on samples in the unlabeled dataset to obtain the corresponding feature layer vector and prediction probability vector; Calculate the marginal uncertainty measure for each sample based on the predicted probability vector; When the selected sample set is empty, the sample with the largest marginal uncertainty measure is added to the selected sample set as the initial uncertainty anchor point; Based on the initial anchor point of uncertainty, calculate the probability space distance and feature space distance between each sample in the unselected sample set and each sample in the selected sample set, and take the minimum value as the probability space diversity component and the feature space diversity component. Perform the following sampling steps: The diversity components in the probability space and the diversity components in the feature space are normalized and then weighted and fused to obtain the diversity contribution of the two spaces. A nonlinear adaptive comprehensive score is constructed based on the marginal uncertainty measure and the contribution of dual-space diversity. The sample selection probability is calculated through the nonlinear adaptive comprehensive score. Based on the sample selection probability, non-repeating sampling is performed and the samples are added to the selected sample set. If the number of samples in the selected sample set does not reach the preset number, add the unselected samples one by one and repeat the sampling step; Once the preset number is reached, submit the code to experts for annotation, and add it to the already annotated training set to drive model retraining.

2. The active learning annotation method based on dual-space adaptive sampling according to claim 1, characterized in that, When the selected sample set is empty, the probability space diversity component is initialized based on the probability space distance of the uncertain initial anchor point; the feature space diversity component is initialized based on the feature space distance of the uncertain initial anchor point.

3. The active learning annotation method based on dual-space adaptive sampling according to claim 1, characterized in that, The marginal uncertainty measure is the maximum value of 1 minus the difference between the predicted probability of the largest category and the predicted probability of the second largest category.

4. The active learning annotation method based on dual-space adaptive sampling according to claim 1, characterized in that, The probability space distance The L1 norm, based on the predicted probability vector, is used as a metric, expressed by the formula: ； in, Indicates the first The sample was predicted as the first... A probability vector for each category; Indicates the first The sample was predicted as the first... A probability vector for each category; This is for absolute value operations; The feature space distance The cosine distance, based on the feature layer vector representation, is used as the metric, expressed by the formula: ； in, Indicates the first Feature layer vectors of each sample; Indicates the first Feature layer vectors of each sample; The norm of a vector.

5. The active learning annotation method based on dual-space adaptive sampling according to claim 1, characterized in that, The contribution of dual-space diversity is obtained by weighting and fusing the probability space diversity component and the feature space diversity component using a fusion coefficient. .

6. The active learning annotation method based on dual-space adaptive sampling according to claim 1, characterized in that, The nonlinear adaptive comprehensive score By measuring marginal uncertainty and the contribution of dual-space diversity The result is obtained by summing after performing a nonlinear transformation, and can be expressed as a formula: ； in, This is a non-linear adjustment parameter.

7. The active learning annotation method based on dual-space adaptive sampling according to claim 1, characterized in that, The sample selection probability is expressed by the formula: ； in, For the first The sample selection probability of each sample; U is the unlabeled dataset; For the first Nonlinear adaptive comprehensive score for each sample.

8. The active learning annotation method based on dual-space adaptive sampling according to claim 1, characterized in that, Feature extraction is performed using convolutional neural networks or attention-based neural networks.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 8.

Images