An image labeling and model training method based on active learning

Abstract

The application discloses an image labeling and model training method based on active learning, which comprises the following steps: firstly, an image recognition task is established to collect image data for training, and a small part is labeled as an initial labeled data set; then, a double-view high-confidence pseudo-label training method is used to train all labeled and unlabeled images, the trained model is used to infer the unlabeled samples, the prediction result and the feature representation of the model are obtained, the consistency between the prediction results of the two networks is used to screen uncertain samples, representative samples are selected through clustering in the feature space, and the annotator is used for labeling; after the representative samples in the current batch are labeled, the representative samples are added to the labeled data set for a new round of training until the total amount of labeling reaches a preset value. The method can overcome the problem of high labeling cost of the existing deep learning-based image recognition model, effectively reduce the required labeling amount, and obtain an accurate image recognition model.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically relating to an image annotation and model training method based on active learning. Background Technology

[0002] Image recognition aims to identify content in images using computer programs, including image classification, object detection, and segmentation. In recent years, deep learning-based image recognition technology has achieved advanced performance. Its success relies on large-scale labeled data, advanced neural network models, and powerful computing capabilities. Large-scale labeled data is a crucial foundation for the high performance of deep learning models. However, in many scenarios, data labeling is time-consuming or costly. For example, in certain domains, accurate image labeling requires experienced experts, or the complexity of images makes labeling extremely time-consuming. Furthermore, in large-scale datasets, different samples contain redundant information; repeatedly labeling redundant data not only increases labeling costs but also reduces efficiency. Therefore, how to label the most valuable samples while maintaining a limited labeling budget is key to reducing labeling costs, improving labeling efficiency, and enhancing model performance.

[0003] To address this issue, using active learning methods for image annotation and model training can effectively reduce annotation costs. During model training, active learning algorithms select the most informative samples for manual annotation based on model performance and data characteristics. Common methods for selecting the most informative samples include those based on sample uncertainty and diversity.

[0004] 1) Uncertainty Estimation-Based Methods: These methods train an AI model on a subset of labeled samples, then use this model to make predictions on unlabeled samples and estimate the uncertainty of the predictions. Common uncertainty estimation methods include calculating entropy based on the probability distribution of the predictions, or using test-time data augmentation to make multiple predictions and then calculating the entropy by averaging the probabilities of the multiple predictions. However, most existing methods estimate uncertainty using a single model. Since a single model is prone to bias during training, it can easily produce results with low uncertainty but incorrect predictions, thus limiting the effectiveness of its uncertainty estimation. Furthermore, samples selected solely based on uncertainty have a certain degree of redundancy. For example, if two similar samples both have high uncertainty, selecting and labeling both samples involves redundant work and does not necessarily yield more benefit than labeling only one of them.

[0005] 2) Representative Sample-Based Methods: These methods map all samples to a feature space using a feature extractor and select representative samples based on clustering (e.g., samples closest to the class centers). This approach effectively reduces redundancy in the selected samples and can obtain more diverse labeled data within a limited labeling budget. However, these methods do not consider the current learning status of the model during sample selection, nor do they consider the model's decision boundaries when selecting samples. Therefore, for a specific model, their sample selection strategy often lacks adaptability, resulting in limited performance improvement.

[0006] Furthermore, existing active learning methods do not fully utilize unlabeled samples for model training and sample selection. Since the amount of labeled samples is typically small, training a model using only currently labeled training samples easily leads to overfitting, and the quality of uncertainty estimation or feature mapping based on this model has limitations. Therefore, there is still a significant performance gap between current active learning algorithms and learning methods that use large-scale labeled data. There is an urgent need for better ways to fully utilize limited labeled data and large amounts of unlabeled data to train better-performing models, thereby improving the selection of samples to be labeled and enhancing the efficiency and quality of the data labeling process. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides an image annotation and model training method based on active learning, overcoming the dependence of existing deep learning-based image recognition methods on large-scale labeled datasets.

[0008] The technical solution adopted in this invention is: an image annotation and model training method based on active learning, the specific steps of which are as follows:

[0009] S1. Establish an image recognition task, collect a batch of image data as training data, use n% (n%≤1%) of samples randomly labeled by experts as the initial labeled dataset, and the remaining samples as the unlabeled dataset to establish the training dataset;

[0010] S2. Using labeled and unlabeled samples, train a pair of deep neural network models using the dual-view high-confidence pseudo-label method, and use class contrast learning loss for feature constraints to obtain the trained model.

[0011] S3. Based on the model trained in step S2, predict on the unlabeled dataset, and obtain the model's prediction results and their feature representations. Then, utilize dual-view... Figure 1 Consistency calculation of uncertainty, screening out samples with high uncertainty;

[0012] S4. Based on the deep neural network model trained in step S2, cluster the uncertain samples selected in step S3 in the feature space, and then select representative samples from each cluster to form a set to be labeled for labelers to label.

[0013] S5. The annotator annotates the current set to be annotated. After the annotation is completed, the newly annotated samples are added to the labeled dataset for a new round of training. That is, steps S2-S4 are repeated until the number of annotated samples reaches the pre-set annotation budget.

[0014] Furthermore, step S2 is specifically as follows:

[0015] Let A and B represent two neural networks with the same structure but different parameters; let f A ,h A ,g A and f B ,h B ,g B These represent the encoder, prediction head, and feature projector of the first and second neural networks, respectively. This indicates data augmentation sets, including: random flipping, rotation, and adding Gaussian noise.

[0016] For an input sample x i The random augmentation was performed twice in parallel, and the result is expressed as follows: and Input the data into network A and network B respectively for prediction. The corresponding prediction result is expressed as follows:

[0017]

[0018]

[0019] in, Represent networks A and B respectively for the augmented x. i The prediction results. For labeled samples, cross-entropy loss is used to supervise the training of the network, as shown in the following expression:

[0020]

[0021] Where C represents the number of categories, y i Indicates sample x i Manually labeled. i,c Indicates x in the label i The probability of belonging to class c. and Let represent the corresponding probability values ​​predicted by networks A and B, respectively. For unlabeled samples, define a high-confidence pseudo-label loss for the two views, using... and The pseudo-label supervision losses for networks A and B are expressed as follows:

[0022]

[0023]

[0024] in, and These represent the pseudo-labels obtained from networks A and B, respectively. On the unlabeled dataset, Used for supervision and Used for supervision L ce This represents the cross-entropy loss function. 1[·] represents a value indicator, which takes the value 1 if its internal expression is true, and 0 otherwise. express The highest probability among all categories. If Exceeding the threshold τ, These are then considered reliable pseudo-labels.

[0025] Then, a class contrast learning loss function is constructed, which is applied through the feature projector g. A and g B To each Features and The feature mapping is given by a latent variable space, as shown in the following expression:

[0026]

[0027] set up Let represent a mini-batch of images, where the subsets of labeled and unlabeled images are denoted as and respectively. and use to indicate The negative sample set, and They represent from and The negative sample in the dataset is defined as follows:

[0028]

[0029]

[0030] in, Indicates sample x i Auxiliary tags. For The value is manually labeled y i ,for The value is its pseudo-label. Then, the class contrastive learning loss function is constructed, and the contrastive learning losses of networks A and B are expressed as follows: and Their definition expressions are as follows:

[0031]

[0032]

[0033] Where T represents the temperature coefficient. Based on the aforementioned supervised loss, dual-view high-confidence pseudo-label loss, and class contrast learning loss, the total loss function for networks A and B in each training round is... The expression is as follows:

[0034]

[0035] Where λ1 and λ2 represent the weights of the pseudo-label loss function and the class contrast learning loss, respectively. Using the total loss function... Train A and B until the model converges, and you will get a trained model.

[0036] Furthermore, step S3 is specifically as follows:

[0037] use This represents all unlabeled samples in the current annotation phase. For a sample to be labeled... use These represent the prediction results of networks A and B, respectively. The one-hot encoding is obtained using the argmax operation. and Will and Inconsistent samples are considered uncertain samples:

[0038]

[0039] Furthermore, step S4 is specifically as follows:

[0040] For D us A sample x i ∈D us By averaging the features extracted by trained networks A and B, we obtain the feature representation. Then, in the feature space, D us Clustering is performed. The K-means algorithm is used for clustering, and the number of clusters is K, where K represents the number of images to be labeled in the current annotation round. The k-th cluster center is represented as O(k).k Based on this, the sample closest to the center within each cluster is selected as the representative sample, as shown in the following expression:

[0041]

[0042] Where ||·|| represents the Euclidean distance. A representative sample set D is selected. cs Then, the annotator will annotate D. cs K samples are labeled and added to the labeled set for use in the next round of model training.

[0043] The beneficial effects of this invention are as follows: The method of this invention first establishes an image recognition task, collects image data for training, and labels a small portion as an initial labeled dataset. Then, a dual-view high-confidence pseudo-label training method is used to incorporate all labeled and unlabeled images for training. The trained model is then used to infer the unlabeled samples, obtaining the model's prediction results and their feature representations. Uncertain samples are then filtered using the consistency between the prediction results of the two networks. Representative samples are selected through clustering in the feature space for labeling. After the representative samples of the current batch are labeled, they are added to the labeled dataset for a new round of training until the total number of labels reaches a preset value. This invention employs an active learning-based approach to achieve efficient image labeling and model training, overcoming the high labeling costs required by existing deep learning-based image recognition models, effectively reducing the required amount of labeling, and obtaining an accurate image recognition model.

[0044] The method of this invention requires only a small number of labeled images, which can greatly reduce the cost of image labeling and achieve high recognition accuracy. It employs a dual-view high-confidence pseudo-label training method to generate pseudo-labels for unlabeled images with high confidence. The model is trained using a mutual supervision approach to enhance its generalization ability, and features are constrained through contrastive learning loss to obtain a feature space more conducive to clustering. The method utilizes a dual-view... Figure 1 Consistent methods select uncertain samples and choose highly representative samples by clustering features, thereby more effectively selecting the most valuable samples to be labeled, and training a model with high accuracy can be achieved with only low labeling costs. Attached Figure Description

[0045] Figure 1 This is a flowchart of an image annotation and model training method based on active learning according to the present invention.

[0046] Figure 2 This is a flowchart illustrating the model training process based on dual-view high-confidence pseudo-labels and category contrastive learning loss in an embodiment of the present invention.

[0047] Figure 3 This is a flowchart illustrating the selection methods for uncertain and representative samples in embodiments of the present invention.

[0048] Figure 4 This is a comparison chart of the model accuracy of the method of the present invention in the annotation process with other methods in this embodiment of the invention. Detailed Implementation

[0049] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0050] This embodiment was implemented on a computer with an Intel(R) Core(TM) i9-9900K 3.60GHz CPU, an NVIDIA GTX3090 GPU, and 24.0GB of memory. The programming language was Python, and the deep learning framework used was PyTorch.

[0051] like Figure 1 The flowchart of an image annotation and model training method based on active learning according to the present invention is shown below. The specific steps are as follows:

[0052] S1. Establish an image recognition task, collect a batch of image data as training data, use n% (n%≤1%) of samples randomly labeled by experts as the initial labeled dataset, and the remaining samples as the unlabeled dataset to establish the training dataset;

[0053] S2. Using labeled and unlabeled samples, train a pair of deep neural network models using the dual-view high-confidence pseudo-label method, and use class contrast learning loss for feature constraints to obtain the trained model.

[0054] S3. Based on the model trained in step S2, make predictions on the unlabeled dataset, and obtain the model's prediction results and their feature representations. Then, utilize dual-view... Figure 1 Consistency (using the consistency between the predictions of two trained network models) is used to calculate uncertainty and filter out samples with high uncertainty.

[0055] S4. Based on the deep neural network model trained in step S2, cluster the uncertain samples selected in step S3 in the feature space, and then select representative samples from each cluster to form a set to be labeled for labelers to label.

[0056] S5. The annotator annotates the current set to be annotated. After the annotation is completed, the newly annotated samples are added to the labeled dataset for a new round of training. That is, steps S2-S4 are repeated until the number of annotated samples reaches the pre-set annotation budget.

[0057] In this embodiment, the CRC5000 colorectal cancer pathology image dataset is used for active learning annotation and image classification model training. In step S1, the CRC5000 public dataset is used, which contains 5000 pathology images from 8 different tissue types. In this embodiment, a total of 10 rounds of annotation are conducted, with 1% of the data (i.e., 50 images) annotated each time, for a total annotation budget of 500 images. Under initial conditions, 1% of the data is annotated by experts as the initial labeled dataset, and the remainder is treated as the unlabeled dataset.

[0058] In this embodiment, step S2 is specifically as follows:

[0059] like Figure 2 As shown, A and B represent two neural networks with the same structure but different parameters, implemented using ResNet50. ; f A ,h A ,g A and f B ,h B ,g B These represent the encoder, prediction head, and feature projector of the first and second neural networks, respectively. This indicates data augmentation sets, including: random flipping, rotation, and adding Gaussian noise.

[0060] For an input sample x i The process involves two parallel random enhancements (including random flipping, rotation, and adding Gaussian noise), and the results are expressed as follows: and Input the data into network A and network B respectively for prediction. The corresponding prediction result is expressed as follows:

[0061]

[0062]

[0063] in, Represent networks A and B respectively for the augmented x. i The prediction results. For labeled samples, cross-entropy loss is used to supervise the training of the network, as shown in the following expression:

[0064]

[0065] Where C represents the number of categories, y i Indicates sample x i Manually labeled. i,c Indicates x in the label i The probability of belonging to class c. and Let represent the corresponding probability values ​​predicted by networks A and B, respectively. To better utilize unlabeled samples, a high-confidence pseudo-label loss for the dual-view model is further defined. and The pseudo-label supervision losses for networks A and B are expressed as follows:

[0066]

[0067]

[0068] in, and These represent the pseudo-labels obtained from networks A and B, respectively. On the unlabeled dataset, Used for supervision and Used for supervision L ce This represents the cross-entropy loss function. 1[·] represents a value indicator, which takes the value 1 if its internal expression is true, and 0 otherwise. express The highest probability among all categories. If If the threshold τ (set to 0.8) is exceeded, These are then considered reliable pseudo-labels.

[0069] To further utilize unlabeled data for feature learning, a class contrastive learning loss function is constructed. This is achieved through a feature projector g. A and g B To each Features and The feature mapping is given by a latent variable space, as shown in the following expression:

[0070]

[0071] Due to the features after projection From the same image x i The two are considered a positive sample pair, while Negative sample pairs are defined as features of labels from other categories obtained in network B. Let... Let represent a mini-batch of images, where the subsets of labeled and unlabeled images are denoted as and respectively. and use to indicate The negative sample set, and They represent from and The negative sample in the dataset is defined as follows:

[0072]

[0073]

[0074] in, Indicates sample x i Auxiliary tags. For The value is manually labeled y i ,for The value is its pseudo-label. Then, the class contrastive learning loss function is constructed, and the contrastive learning losses of networks A and B are expressed as follows: and Their definition expressions are as follows:

[0075]

[0076]

[0077] Where T represents the temperature coefficient, set to 0.1. Based on the aforementioned supervised loss, dual-view high-confidence pseudo-label loss, and class contrast learning loss, the total loss function for networks A and B in each training round is... The expression is as follows:

[0078]

[0079] Where λ1 and λ2 represent the weights of the pseudo-label loss function and the class contrast learning loss, respectively; in this embodiment, λ1 = 0.5 and λ2 = 0.1. Using the total loss function... Train A and B until the model converges, and you will get a trained model.

[0080] like Figure 3 As shown, in this embodiment, step S3 is specifically as follows:

[0081] use This represents all unlabeled samples in the current annotation phase. For a sample to be labeled... use These represent the prediction results of networks A and B, respectively. The one-hot encoding is obtained using the argmax operation. and Will and Inconsistent samples are considered uncertain samples:

[0082]

[0083] like Figure 3 As shown, in this embodiment, step S4 is specifically as follows:

[0084] For D us A sample x i ∈D us By averaging the features extracted by trained networks A and B, we obtain the feature representation. Then, in the feature space, D us Clustering is performed. The K-means algorithm is used for clustering, and the number of clusters is K = 50, where K represents the number of images to be labeled in the current annotation round. The k-th cluster center is represented as O(k). k Based on this, the sample closest to the center within each cluster is selected as the representative sample, as shown in the following expression:

[0085]

[0086] Where ||·|| represents the Euclidean distance. A representative sample set D is selected. cs Then, the annotator will annotate D. cs K samples are labeled and added to the labeled set for use in the next round of model training.

[0087] In this embodiment, in step S5, the annotator annotates the current set D to be annotated. cs Perform labeling, and repeat steps S2-S4 after labeling, iterating 9 times until the number of labeled samples reaches the pre-set labeling budget (50 x 10 = 500).

[0088] Figure 4 This graph compares the classification results of the method of this invention on the CRC5000 colorectal cancer pathology image dataset with other methods. The horizontal axis represents the number of annotations in different training epochs, and the vertical axis represents the accuracy. In the graph, diamond-shaped dots represent randomly selected samples for annotation, serving as the baseline method; circular dots represent a representative sample selection method based on Washington distance; "×"-shaped dots represent a sample selection method based on uncertainty estimation using image augmentation; triangles represent the results of the method of this invention; and dashed lines without annotations represent the method of training the neural network using all labeled data. It can be seen that, under different annotation ratios, the model performance obtained by the method of this invention significantly outperforms other methods.

[0089] In summary, most current deep learning-based image classification methods rely on large-scale labeled datasets, which are time-consuming, labor-intensive, and costly. The method of this invention, based on active learning, requires only a small number of labeled images, significantly reducing image labeling costs and achieving high recognition accuracy. Most current active learning methods neglect the utilization of unlabeled images. To address this, this invention proposes a dual-view high-confidence pseudo-label training method. Pseudo-labels are generated for high-confidence unlabeled images, and the model is trained using mutual supervision to enhance its generalization ability. Furthermore, feature constraints are applied through contrastive learning loss to obtain a feature space more conducive to clustering. Most existing active learning methods only focus on uncertainty or representativeness when selecting strategies. This invention proposes a dual-view... Figure 1 Consistent methods select uncertain samples and choose highly representative samples by clustering features, thereby more effectively selecting the most valuable samples to be labeled, and training a model with high accuracy can be achieved with only low labeling costs.

[0090] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the implementation methods of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the present invention.

Claims

1. An image annotation and model training method based on active learning, the specific steps of which are as follows: S1. Establish an image recognition task, collect a batch of image data as training data, use n% (n%≤1%) of samples randomly labeled by experts as the initial labeled dataset, and the remaining samples as the unlabeled dataset to establish the training dataset; S2. Using labeled and unlabeled samples, train a pair of deep neural network models using the dual-view high-confidence pseudo-label method, and use class contrast learning loss for feature constraints to obtain the trained model. S3. Based on the model trained in step S2, make predictions on the unlabeled dataset, and obtain the model's prediction results and feature representations. Then, use the dual-view consistency to calculate uncertainty and filter out samples with high uncertainty. S4. Based on the deep neural network model trained in step S2, cluster the uncertain samples selected in step S3 in the feature space, and then select representative samples from each cluster to form a set to be labeled for labelers to label. S5. The annotator annotates the current set to be annotated. After the annotation is completed, the newly annotated samples are added to the labeled dataset for a new round of training. That is, steps S2-S4 are repeated until the number of annotated samples reaches the pre-set annotation budget.

2. The image annotation and model training method based on active learning according to claim 1, characterized in that, Step S2 is as follows: Let A and B represent two neural networks with the same structure but different parameters; let f A ,h A ,g A and f B ,h B ,g B These represent the encoder, prediction head, and feature projector of the first and second neural networks, respectively; using Data augmentation includes: random flipping, rotation, and adding Gaussian noise; For an input sample x i The random augmentation was performed twice in parallel, and the result is expressed as follows: and The data is input into network A and network B respectively for prediction; using... The corresponding prediction result is expressed as follows: in, Represent networks A and B respectively for the augmented x. i The prediction results; for labeled samples, cross-entropy loss is used to supervise the training of the network, as shown in the following expression: Where C represents the number of categories, y i Indicates sample x i Manually labeled; y i,c Indicates x in the label i The probability of belonging to class c; and Let A and B represent the corresponding probability values ​​predicted by networks A and B, respectively; for unlabeled samples, define a high-confidence pseudo-label loss for the two views, using... and The pseudo-label supervision losses for networks A and B are expressed as follows: in, and These represent the pseudo-labels obtained from networks A and B, respectively; on the unlabeled dataset, Used for supervision and Used for supervision L ce represents the cross-entropy loss function; 1[·] represents a value indicator that takes the value 1 if its internal expression is true, and 0 otherwise; express The highest probability among all categories; if Exceeding the threshold τ, These are considered reliable pseudo-labels; Then, a class contrast learning loss function is constructed, which is applied through the feature projector g. A and g B To each Features and The feature mapping is given by a latent variable space, as shown in the following expression: set up Let represent a mini-batch of images, where the subsets of labeled and unlabeled images are denoted as and respectively. and use to indicate The negative sample set, and They represent from and The negative sample in the dataset is defined as follows: in, Indicates sample x i Auxiliary tags; for The value is manually labeled y i ,for The value is its pseudo-label. Then, the class contrastive learning loss function is constructed, and the contrastive learning losses of networks A and B are expressed as follows: and Their definition expressions are as follows: Where T represents the temperature coefficient; based on the above-mentioned supervised loss, dual-view high-confidence pseudo-label loss, and class contrast learning loss, the total loss function of networks A and B in each round of training is... The expression is as follows: Where λ1 and λ2 represent the weights of the pseudo-label loss function and the class contrast learning loss, respectively; using the total loss function Train A and B until the model converges, and you will get a trained model.

3. The image annotation and model training method based on active learning according to claim 1, characterized in that, Step S3 is as follows: use This represents all unlabeled samples in the current annotation phase; for a sample to be labeled... use These represent the prediction results of networks A and B, respectively. The one-hot encoding is obtained using the argmax operation. and Will and Inconsistent samples are considered uncertain samples: 。 4. The image annotation and model training method based on active learning according to claim 1, characterized in that, Step S4 is as follows: For D us A sample x i ∈D us By averaging the features extracted by trained networks A and B, we obtain the feature representation. Then, in the feature space, D us Clustering is performed; the K-means algorithm is used for clustering, and the number of clusters is K, where K represents the number of images to be labeled in the current annotation round; the k-th cluster center is represented by O(k). k Based on this, the sample closest to the center within each cluster is selected as the representative sample, as shown in the following expression: Where ||·|| represents the Euclidean distance; select a representative sample set D. cs Then, the annotator will annotate D. ccs K samples are labeled and added to the labeled set for use in the next round of model training.

Images