Pseudo-label based re-weighting semi-supervised image classification method

Abstract

The present application relates to semi-supervised learning technology, image classification field, for putting forward a kind of based on pseudo label's re-weighting semi-supervised image classification technology, utilize in learning process, the importance of each class to model is different, fully learn different class, the present application, based on pseudo label's re-weighting semi-supervised image classification method, steps are as follows: the image data set for training is handled;Supervised loss in the e th iteration is calculated using labeled data;Unsupervised loss in the e th iteration is calculated using unlabeled data;The overall loss in the e th iteration of model is calculated;Model parameter is updated using gradient descent method;According to the condition that pseudo label is generated in the e th iteration process of model, the weight coefficient of each class is calculated;When testing, the original sample is sent into the model of training completion, and the prediction result of sample is output by model, and the class of the highest probability is the class of sample.The present application is mainly applied to image classification occasion.

Description

Technical Field

[0001] This invention relates to semi-supervised learning techniques and the field of image classification, specifically to a reweighted semi-supervised image classification method based on pseudo-labels. Background Technology

[0002] In recent years, the field of deep learning has achieved tremendous success, and a key factor in its rapid development is the availability of massive amounts of training data. However, in practical applications, labeling datasets is expensive. Since obtaining unlabeled data is relatively simple, we often have partially labeled data and large amounts of unlabeled data. Semi-supervised learning focuses on how to effectively utilize large amounts of unlabeled data, aiming to reduce the need for labeled data. Currently, semi-supervised learning has been successfully applied to various real-world tasks (e.g., image classification, object detection, and semantic segmentation) and is receiving increasing attention.

[0003] Image classification is a fundamental topic in computer vision and a crucial basis for applications such as object detection and face recognition. Therefore, image classification technology has high research and application value. As the name suggests, image classification involves a model predicting the label information of an image by mining the feature information contained within it, thus classifying the image. Image classification has been successfully applied to various aspects of human life. A key factor in the rapid development of deep learning is the availability of large datasets. Traditional deep learning can be divided into supervised learning, unsupervised learning, and semi-supervised learning. In practical applications, although we can easily obtain large amounts of data, the high-quality labeled data required for supervised learning remains expensive, requiring significant human and material resources. For example, in spam classification, the number of emails is enormous, and users cannot check each email individually to determine if it is spam. Labeled data contains valuable label information, and unlabeled data also contains rich information waiting to be mined. Therefore, developing relatively abundant unlabeled data is crucial and has attracted increasing attention. Because labeled datasets are very expensive, multi-datasets consist of a small amount of labeled data and a large amount of unlabeled data. The way to train a model based on such data is called semi-supervised learning, which aims to reduce the need for labeled data by making full use of the large amount of unlabeled data.

[0004] In semi-supervised classification tasks, two effective techniques are widely used in algorithms: pseudo-labeling and consistency regularization. Pseudo-label-based models, also known as self-training, are a common strategy in semi-supervised learning. They aim to generate pseudo-labels for unlabeled data by the model itself to guide the training process. Typically, pseudo-label-based models select predictions with high confidence as supervisory information to be added to the training process. Consistency regularization-based models assume that predictions will not change after perturbations are added to the data. Both techniques have achieved quite good performance. Recently, some advanced algorithms, such as MixMatch and FixMatch, have integrated these techniques and achieved state-of-the-art performance.

[0005] A common characteristic of these semi-supervised classification models is that they rely solely on a predefined threshold to select high-confidence pseudo-labels. An inherent problem with these algorithms is that they ignore the class distribution of the selected pseudo-labels across different categories, especially in the initial stages of training, where only a small number of pseudo-labels have confidence levels above the predefined threshold. For underlearned classes, fewer pseudo-labels are retained because the model has lower confidence in its predictions for these classes, leading to an imbalanced class distribution of retained pseudo-labels. Furthermore, despite differences in learning difficulty across different categories, most existing semi-supervised models treat all categories equally. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention aims to propose a pseudo-label-based reweighted semi-supervised image classification technique. This technique leverages the fact that each class has varying importance to the model during the learning process, allowing for the comprehensive learning of different classes. Therefore, the technical solution adopted by this invention is a pseudo-label-based reweighted semi-supervised image classification method, with the following steps:

[0007] 101: Process the image dataset used for training by dividing the entire dataset into labeled datasets. and unlabeled datasets The same number of data points are randomly drawn from each category. These data points and their labels constitute labeled data, and the remaining data constitutes unlabeled dataset. This means that the labels of these data points were not added to the unlabeled dataset.

[0008] 102: Utilizing labeled data Calculate the supervised loss L in the e-th iteration. (e，s) ;

[0009] 103: Utilizing Unlabeled Data Calculate the unsupervised loss L in the e-th iteration. (e，u) ;

[0010] 104: Calculate the overall loss L in the e-th iteration of the model.e

[0011] L e =L (e，s) +λL (e，u)

[0012] Where λ is a hyperparameter used to balance the contributions of supervised and unsupervised losses;

[0013] 105: Update model parameters using gradient descent;

[0014] 106: Calculate the weight coefficients for each category based on the pseudo-labels generated by the model during the e-th iteration;

[0015] 107: During testing, the original samples are fed into the trained model f(), and the model outputs the prediction results for the samples, where the class with the highest probability is the class of the sample.

[0016] The preprocessing in step 101 mainly includes the following steps:

[0017] 1011: Randomly select the same number of samples from each category from the original dataset, retain their labels, and use the selected samples and their labels as the labeled dataset;

[0018] 1012: The labels of the remaining unsampled samples are discarded, and the samples are kept separately as an unlabeled dataset;

[0019] 1013: Labeled and unlabeled datasets together constitute the dataset used to train the model.

[0020] The supervised loss calculation process in step 102 mainly includes the following steps: 1021: Randomly select N labeled samples l The dataset consists of labeled samples {(x) i y i ): i = 1, ..., N l}, for sample x i Perform a weak enhancement operation W() to generate a weak enhancement function W(x). i ), where W() is the standard flip and shift enhancement strategy, which randomly flips the original image with a probability of 0.5;

[0021] 1022: W(x) i The data is fed into the neural network f(), and the network outputs a predicted distribution P. i =f(W(x) i There is a supervised loss L (e，s) For P i With y i Cross-entropy between:

[0022]

[0023] Where CE() is the cross-entropy function.

[0024] The calculation of unsupervised loss in step 103 mainly includes the following steps:

[0025] 1031: Randomly select N u The dataset consists of unlabeled samples. For the sample Generate by performing weak enhancement operations

[0026] 1032: Will The data is fed into the neural network f(), and the network outputs a predicted distribution.

[0027] 1033: Based on q i Unlabeled data Generating pseudo-labels, specifically, if the probability of the most likely class in the prediction results is higher than a threshold τ, i.e., max(q) i If )≥τ, then retain its pseudo-label.

[0028] 1034: For unlabeled data that retains pseudo-labels Generate by performing a strong augmentation operation S() on the original data. S() is an automatic data augmentation method that defines a set of 14 operations: Identity, AutoContrast, Equalize, Rotate, Solarize, Color, Posterize, Contrast, Brightness, Sharpness, ShearX, ShearY, TranslateX, and TranslateY. RandAugmmt randomly selects two operations from this set for each strong augmentation operation, with an increment of 10.

[0029] 1035: Will The data is fed into the neural network f(), and the network outputs a predicted distribution.

[0030] 1036: Calculate the weighted unsupervised loss L in the e-th iteration based on the weight coefficients. (e，u) :

[0031]

[0032] Where ω e-1(c) represents the weight coefficients of each category in the previous round, which are updated after each round of iterative training.

[0033] The process of calculating the weight coefficients for each category in step 106 mainly includes the following steps:

[0034] 1061: Calculate the model's learning state for each category based on the pseudo-labels generated by the model. For example, the model's learning state σ for category c. e (c) is:

[0035] σ e (c)=>I(max(q i )≥τ)·I(arg max(q i ))=c

[0036] Where I() is an indicator function, which outputs 1 when the input is true and 0 otherwise;

[0037] 1062: Based on the learning state σ e (c) Calculate the relative learning state factor for each category, for example, the learning state factor β for category c. e (c) is:

[0038]

[0039] 1063: Based on the learning state factor β e (c) Calculate the weight coefficient for each category. For example, the weight coefficient for category c is ω. e (c) is:

[0040] ω e (c)=k-β e (c)

[0041] Where k is used for scaling ω e (c) Range of hyperparameters.

[0042] The features and beneficial effects of this invention are:

[0043] 1. This invention is a pseudo-label-based reweighted semi-supervised image classification technique that takes into account the learning difficulty between different categories in semi-supervised classification.

[0044] 2. This invention encourages models to pay more attention to categories with fewer pseudo-labels during the learning phase, and can be combined with any pseudo-label-based semi-supervised algorithm to improve the performance of semi-supervised learning.

[0045] 3. The present invention has achieved optimal results on multiple standard datasets (such as CIFAR-10, CIFAR-100, SVHN and STL-10). Attached image description:

[0046] Figure 1 This is a flowchart illustrating the training of the model and the use of the model to predict classification results in this invention. Detailed Implementation

[0047] The purpose of this invention is to address the problem of imbalanced pseudo-label distribution in current semi-supervised classification tasks by proposing a pseudo-label-based reweighted semi-supervised image classification technique. This invention considers the learning difficulty between different classes in semi-supervised classification. Specifically, the learning process is dynamically adjusted according to the model's current learning state, encouraging the model to pay more attention to classes with fewer pseudo-labels, rather than treating every sample equally. In other words, during the learning process, each class has different importance to the model, thus allowing the model to fully learn different classes.

[0048] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0049] To address the above problems, this invention proposes a reweighted semi-supervised image classification method based on pseudo-labels. This method considers the learning difficulty between different categories in semi-supervised classification. Specifically, based on the model's current learning state, this invention dynamically adjusts the learning process, encouraging the model to pay more attention to the class with fewer pseudo-labels, rather than treating every sample equally.

[0050] Example 1

[0051] A reweighted semi-supervised image classification method based on pseudo-labels, comprising the following steps:

[0052] 101: Process the image dataset used for training by dividing the entire dataset into labeled datasets. and unlabeled datasets Specifically, an equal number of data points are randomly selected from each category; these data points and their labels constitute the labeled data. The remaining data form the unlabeled dataset, meaning that the labels for these data points were not added to the unlabeled dataset.

[0053] The preprocessing process in step 101 mainly includes the following steps:

[0054] 1011: Randomly select the same number of samples from each category from the original dataset, retain their labels, and use the selected samples and their labels as the labeled dataset;

[0055] 1012: The labels of the remaining unsampled samples are discarded, and the samples are kept separately as an unlabeled dataset;

[0056] 1013: Labeled and unlabeled datasets together constitute the dataset used to train the model.

[0057] 102: Utilizing labeled data Calculate the supervised loss L(e, s) in the e-th iteration;

[0058] The supervised loss calculation process in step 102 mainly includes the following steps:

[0059] 1021: Randomly select N l A dataset {(xi, y)} consists of labeled samples. i ): i = 1, ..., Nl}, perform a weak enhancement operation W() on sample xi to generate W(xi), where W() is a standard flip and shift enhancement strategy, randomly flipping the original image with a probability of 0.5.

[0060] 1022: W( xi The data is fed into the neural network f(), and the network outputs a predicted distribution P. i =f(W( xi There is supervised loss L( e，s ) is P i With y i Cross-entropy between:

[0061]

[0062] Where CE() is the cross-entropy function;

[0063] 103: Utilizing Unlabeled Data Calculate the unsupervised loss L(e, u) in the e-th iteration;

[0064] The calculation of unsupervised loss in step 103 mainly includes the following steps:

[0065] 1031: Randomly select Nu unlabeled samples to form a dataset. For the sample Generate by performing weak enhancement operations

[0066] 1032: Will The data is fed into the neural network f(), and the network outputs a predicted distribution.

[0067] 1033: Based on qi being unlabeled data Generating pseudo-labels: Specifically, if the probability of the most likely class in the prediction results is higher than the threshold τ, i.e., max(qi) ≥ τ, then the pseudo-label is retained.

[0068] 1034: For unlabeled data that retains pseudo-labels Generate by performing a strong augmentation operation S() on the original data. S() is an automatic data augmentation method that defines a set of 14 operations: Identity, AutoContrast, Equalize, Rotate, Solarize, Color, Posterize, Contrast, Brightness, Sharpness, ShearX, ShearY, TranslateX, and TranslateY. In this invention, each time a strong augmentation operation is performed, two operations are randomly selected from the above set for use, with an increment of 10.

[0069] 1035: Will The data is fed into the neural network f(), and the network outputs a predicted distribution.

[0070] 1036: Calculate the weighted unsupervised loss L in the e-th iteration based on the weight coefficients. (e，u) :

[0071]

[0072] Where ω e-1 (c) represents the weight coefficients of each category in the previous round, which are updated after each round of training iterations.

[0073] 104: Calculate the overall loss L in the e-th iteration of the model. e :

[0074] L e =L (e，s) +λL (e，u)

[0075] Where λ is a hyperparameter used to balance the contributions of supervised and unsupervised losses;

[0076] 105: Update model parameters using gradient descent;

[0077] 106: Calculate the weight coefficients for each category based on the pseudo-labels generated by the model during the e-th iteration;

[0078] The process of calculating the weight coefficients for each category in step 106 mainly includes the following steps:

[0079] 1061: Calculate the model's learning state for each category based on the pseudo-labels generated by the model. For example, the model's learning state σ for category c. e (c) is:

[0080] σ e (c)=∑I(max(q i )≥τ)·I(arg max(q i ))=c

[0081] Where I() is an indicator function, which outputs 1 when the input is true and 0 otherwise;

[0082] 1062: Based on the learning state σ e (c) Calculate the relative learning state factor for each category, for example, the learning state factor β for category c. e (c) 1 for:

[0083]

[0084] 1 Learning status of category c in round e

[0085] 1063: Based on the learning state factor β e (c) Calculate the weight coefficient for each category. For example, the weight coefficient for category c is ω. e (c) is:

[0086] ω e (c)=k-β e (c)

[0087] Where k is used for scaling ω e (c) Range of hyperparameters;

[0088] 107: During testing, the original samples are fed into the trained model f(), and the model outputs the prediction results for the samples, where the class with the highest probability is the class of the sample.

[0089] 108: Experimental details on the CIFAR-10, CIFAR-100, SVHN, and STL-10 datasets are as follows:

[0090] 1081: Experimental Environment. The hardware configuration consisted of 4 x Titan X Pascal GPUs, the programming language and version was Python 3.8, the open-source environment and version was Anaconda3, the deep learning framework was PyTorch, and the versions of torch and torchvision were 1.4 and 0.5, respectively.

[0091] 1082: Experimental Background. In practical applications of image classification, labeling datasets is expensive. Since obtaining unlabeled data is relatively simple, we often have some labeled data and a large amount of unlabeled data. Semi-supervised image classification uses these datasets to train a model, aiming to fully mine and utilize the information contained in the unlabeled data with the help of a small amount of labeled data, thereby enabling the model to learn better semantic information and improve classification performance.

[0092] 1083: Data Sources. The CIFAR-10, CIFAR-100, SVHN, and STL-10 datasets used in the experiments are all publicly available RGB image datasets. The deep learning framework PyTorch provides interfaces for these datasets, and the datasets can be loaded using the torch.utils.data.Dataset class in the experiments.

[0093] 1084: Experimental Procedure. First, hardware computing resources and a software development environment were acquired, including a GPU and Anaconda environment. Then, code development was performed using the PyTorch deep learning framework, including building the network structure, reading training data, preprocessing the data, and setting training parameters. Finally, the trained model was evaluated using a test set and compared with comparative methods.

[0094] 1085: Comparison Method. We selected FixMatch, a relatively advanced semi-supervised classification algorithm, as the benchmark method for comparison.

[0095] 1086: Experimental Results. The evaluation results of the present invention and the comparative method on the CIFAR-10, CIFAR-100, SVHN, and STL-10 datasets are shown in the table below. The data shown in the table are classification accuracy, where "labels" represents the number of labeled data in the dataset. The experimental results show that the semi-monitored model (BPL) of the present invention has a higher classification accuracy than the comparative method.

[0096]

[0097]

[0098] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

[0099] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0100] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A reweighted semi-supervised image classification method based on pseudo-labels, characterized by the following steps: as follows: 101: Process the image dataset used for training by dividing the entire dataset into labeled datasets. and unlabeled datasets The same number of data points are randomly selected from each category. These data points and their labels constitute labeled data, and the remaining data constitutes unlabeled dataset. This means that the labels of these data points were not added to the unlabeled dataset. 102: Utilizing labeled data Calculate the supervised loss in the e-th iteration. ; 103: Utilizing Unlabeled Data Calculate the unsupervised loss in the e-th iteration. ; 104: Calculate the overall loss in the e-th iteration of the model. ； in It is a hyperparameter used to balance the contributions of supervised and unsupervised losses; 105: Update model parameters using gradient descent; 106: Calculate the weight coefficients for each category based on the pseudo-labels generated by the model during the e-th iteration. The process of calculating the weight coefficients for each category mainly includes the following steps: 1061: Calculate the model's learning state for each category based on the pseudo-labels generated by the model. The model's learning state for category c. for: ； in This is an indicator function, meaning it outputs 1 when the input is true and 0 otherwise. 1062: Based on learning status Calculate the relative learning state factor for each category, and the learning state factor for category c. for: 1063: Based on learning state factors Calculate the weight coefficients for each category. The weight coefficient for category c is: for: ； in It is used for scaling Hyperparameters of the range; 107: During testing, the original samples are fed into the trained model. In this model, the output is the prediction result for the sample, and the class with the highest probability is the class of the sample.

2. The pseudo-label-based reweighted semi-supervised image classification method as described in claim 1, characterized in that, The preprocessing in step 101 mainly includes the following steps: 1011: Randomly select the same number of samples from each category from the original dataset, retain their labels, and use the selected samples and their labels as the labeled dataset; 1012: The labels of the remaining unsampled samples are discarded, and the samples are kept separately as an unlabeled dataset; 1013: Labeled and unlabeled datasets together constitute the dataset used to train the model.

3. The pseudo-label-based reweighted semi-supervised image classification method as described in claim 1, characterized in that, The supervised loss calculation process in step 102 mainly includes the following steps: 1021: Number of randomly selected labeled samples The dataset consists of labeled samples. For the sample Perform weak enhancement operations Generate weak enhancement functions ,in The standard flip and shift enhancement strategy involves randomly flipping the original image with a probability of 0.

5. 1022: will Feed into neural network In the middle, the network output prediction distribution There is a loss of supervision. for and Cross-entropy between: ; in This is the cross-entropy function.

4. The pseudo-label-based reweighted semi-supervised image classification method as described in claim 1, characterized in that, The calculation of unsupervised loss in step 103 mainly includes the following steps: 1031: Random Selection The dataset consists of unlabeled samples. For the sample Generate by performing weak enhancement operations ; 1032: Will Feed into neural network In the middle, the network output prediction distribution ; 1033: Based on Unlabeled data Generate pseudo-labels, specifically, if the probability of the most likely class in the prediction results is higher than a threshold. ,Right now Then retain its pseudo-tags. ; 1034: For unlabeled data that retains pseudo-labels Perform strong augmentation operations on the raw data generate , It is an automatic data augmentation method that sets up an operation set of 14 operations, including: Identity, AutoContrast, Equalize, Rotate, Solarize, Color, Posterize, Contrast, Brightness, Sharpness, ShearX, ShearY, TranslateX, TranslateY, and RandAugment. Each time a strong augmentation operation is performed, two operations are randomly selected from the above operation set for use, with an increment of 10. 1035: Will Feed into neural network In the middle, the network output prediction distribution ; 1036: Calculate the weighted unsupervised loss in the e-th iteration based on the weight coefficients. : ； in These are the weight coefficients for each category in the previous round, which are updated after each iteration of training.

Images