A Semi-Supervised Image Classification Method Based on Evidence Theory

Abstract

This invention discloses a semi-supervised image classification method based on evidence theory, comprising: processing the image dataset used for semi-supervised learning by randomly selecting a small number of samples of the same quantity from each category and retaining the labels of the small number of samples to form a labeled dataset used for training; adding different degrees of perturbation to the original images through data augmentation techniques; performing weak augmentation on the labeled samples and training the model in a supervised manner using the weakly augmented samples; performing weak and strong augmentation on the unlabeled samples respectively, feeding the weakly augmented unlabeled samples into each sub-network to generate evidence, obtaining the uncertainty of each sub-network by modeling through Dirichlet distribution, fusing the obtained evidence and uncertainty using evidence theory to obtain fused evidence, and obtaining more reliable classification results and pseudo-labels based on the fused evidence; training the network using the unlabeled samples and their pseudo-labels, and continuing to classify the images using the trained network.

Description

Technical Field

[0001] This invention relates to the field of image classification, and more particularly to a semi-supervised image classification method based on evidence theory. Background Technology

[0002] In recent years, with the development of deep neural networks and the collection of large-scale data, the field of machine learning has achieved tremendous success. Traditional machine learning can be divided into supervised learning, unsupervised learning, and semi-supervised learning. Supervised learning is a natural learning approach that trains models on labeled datasets, where labels provide supervisory information. Supervised learning can achieve good performance and generalization ability even with large amounts of labeled data. Unsupervised learning trains on unlabeled datasets, learning directly from the raw data and uncovering valuable information hidden within it. Unsupervised learning tasks typically include unsupervised feature learning, probability density estimation, and clustering, and it is also frequently used for pre-training tasks. In practical applications, although large amounts of data are readily available, the high-quality labeled data required for supervised learning remains expensive, requiring significant human and material resources. Therefore, many datasets consist of a small amount of high-quality labeled data and a large amount of unlabeled data. Labeled data contains valuable label information, while unlabeled data also contains rich information waiting to be mined. Therefore, developing relatively abundant unlabeled data is crucial and has attracted increasing attention. Learning from data consisting of a small portion of labeled data and a large portion of unlabeled data is called semi-supervised learning. Its purpose is to reduce the need for labeled data by making full use of the large amount of unlabeled data. The focus of semi-supervised learning is on how to effectively utilize large amounts of unlabeled data.

[0003] In practical applications, there is typically a large amount of unlabeled data and a small subset of high-quality labeled data available. Learning from this data is called semi-supervised learning, which aims to reduce the need for labeled data by fully utilizing the large amount of unlabeled data. Recently, in image classification, representative semi-supervised learning techniques, including pseudo-labeling and consistency regularization, have demonstrated effectiveness and robustness. Pseudo-labeled models are first trained on labeled data and then use predictions on unlabeled data as supervision for continuous training. Consistency regularization methods assume that small perturbations do not change their patterns, thus applying reinforcement techniques and constraining their predictions to be consistent. Semi-supervised learning has been successfully applied to various real-world tasks (e.g., image classification, object detection, and semantic segmentation) to address the lack of labeled data.

[0004] The most popular semi-supervised learning methods now combine pseudo-labels and consistency regularization. For example, MixMatch (Berthelot,D.,Carlini,N.,Goodfellow,I.,Papernot,N.,Oliver,A.,Raffel,CA:Mixmatch:A holistic approach to semi-supervised learning.Advances in Neural Information Processing Systems 32(2019)) randomly augments the original unlabeled image multiple times and averages the predictions to generate pseudo-labels. ReMixMatch (Berthelot,D.,Carlini,N.,Cubuk,ED,Kurakin,A.,Sohn,K.,Zhang,H.,Raffel,C.:Remixmatch:Semi-supervised learning with distribution matching and augmentation anchoring.In:International Conference on Learning Representations(2019)) improves MixMatch based on two assumptions: distribution alignment and augmentation anchoring. Specifically, the label distribution of labeled data is used to align unlabeled data, which helps reduce prediction noise. While the augmentation anchor assumption implies that weak augmentations are more reliable and accurate than strong augmentations, weak augmentation predictions are used as label anchors to guide the learning of strong augmentations. FixMatch (Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, CA, Cubuk, ED, Kurakin, A., Li, CL: Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in Neural Information Processing Systems 33 (2020)) is another, more concise version and highly effective. However, due to the addition of noisy labels during training, predictions are not always accurate, so these heuristic semi-supervised learning methods using pseudo-labels still suffer from confirmation bias. Summary of the Invention

[0005] This invention provides a semi-supervised image classification method based on evidence theory. Addressing the confirmation bias problem in current semi-supervised learning, it proposes an end-to-end semi-supervised image classification technique based on evidence theory. By introducing Dempster–Shafer theory, it effectively integrates the outputs of different sub-networks, thereby making more reliable decisions and reducing the risk of erroneous predictions. This solves the confirmation bias problem in self-training methods and the confidence conservatism problem in ordinary ensemble methods. See the description below for details:

[0006] A semi-supervised image classification method based on evidence theory, the method comprising:

[0007] The image dataset used for semi-supervised learning is processed by randomly selecting a small number of samples of the same number from each category and retaining the labels of the small number of samples to form the labeled dataset used for training.

[0008] Data augmentation techniques are used to add perturbations of varying degrees to the original image; weak augmentation is applied to labeled samples, and the model is trained using the weakly augmented samples in a supervised learning manner.

[0009] Weak and strong enhancements are applied to unlabeled samples respectively. The weakly enhanced version of the unlabeled sample is fed into each sub-network to generate evidence. The uncertainty of each sub-network can be obtained by modeling through Dirichlet distribution. The evidence and uncertainty obtained by fusing evidence theory are used to obtain fused evidence. Based on the fused evidence, more reliable classification results and pseudo-labels can be obtained.

[0010] The network is trained using unlabeled samples and their pseudo-labels, and the trained network is then used to classify images.

[0011] Specifically, the process involves feeding weakly enhanced unlabeled samples into each sub-network to generate evidence. The uncertainty of each sub-network can be obtained through Dirichlet distribution modeling. The evidence and uncertainty obtained are then fused using evidence theory to obtain fused evidence. Based on this fused evidence, more reliable classification results and pseudo-labels can be obtained.

[0012] The enhanced sample W(x) u The data is fed into each branch network to obtain different output evidence;

[0013] The parameters of the Dirichlet distribution are obtained from the output evidence of the branch network; the Dirichlet normalization factor is obtained from the parameters of the Dirichlet distribution; and the belief mass for each class is obtained using the Dirichlet normalization factor and the parameters of the Dirichlet distribution. Uncertainty in branch network prediction and

[0014] Using evidence fusion theory to fuse all branch networks and u k To obtain reliable b and u, the fusion process when k=2 is as follows:

[0015]

[0016]

[0017] in, These are parameters used for normalization. When k ≥ 3, they are merged pairwise in a merge manner. The information belonging to the m-th class extracted for the first branch network. Information belonging to the nth class extracted for the second branch network;

[0018] The α used for classification prediction is obtained by fusing b and u, and the calculation process is as follows:

[0019]

[0020] e = [b1×S,b2×S,...,b M ×S],

[0021] α=[e1+1,e2+1,...,e M +1];

[0022] Among them, b M For the merged information belonging to the m-th class, e M The evidence belonging to class m is derived from the fused information, where M is the total number of classes in the classification task, S is the Dirichlet normalization factor, and α is the parameter of the Dirichlet distribution.

[0023] The classification probability is obtained by applying the softmax activation function to the obtained α, and the pseudo-label with one-hot encoding is obtained from the classification probability.

[0024] The beneficial effects of the technical solution provided by this invention are:

[0025] (1) This invention proposes a simple and effective semi-supervised learning framework based on Dempster-Shafer theory. The method uses multiple differentiated branch networks to jointly predict samples, so that each branch network is influenced by other branches during learning and does not only consider its own prediction, thereby solving the confirmation bias problem.

[0026] (2) This invention proposes an end-to-end image classification method. Using the above method, information carried by unlabeled data can be fully explored and utilized when label data is insufficient, thereby obtaining a more accurate decision boundary in the data space.

[0027] (3) This invention proposes an integration strategy based on Dempster-Shafer theory. This method characterizes the output of each branch network as a Dirichlet distribution and generates the uncertainty of the prediction of each branch network. Then, the results of all branch networks are integrated using the Dempster-Shafer fusion rule to obtain more reliable prediction results.

[0028] (4) This invention introduces the Dempster-Shafer theory into the field of semi-supervised image classification, and solves the problem of overly conservative prediction confidence in ordinary ensemble methods by adding uncertainty to network prediction.

[0029] (5) This invention proposes a simple yet effective image classification model, achieving state-of-the-art results on image datasets CIFAR-10, CIFAR-100, SVHN, and STL-10. For example, on the STL-10 dataset, the proposed model reduces the image classification error rate by 4.7%, meeting the needs of practical applications. Attached Figure Description

[0030] Figure 1 This is a framework diagram of the present invention (taking the setting of two branch networks as an example);

[0031] Figure 2 This is a flowchart illustrating the training of the model and the use of the model to predict classification results in this invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0033] To address the aforementioned problems, this invention proposes a semi-supervised image classification method based on evidence theory. Building upon previous work combining pseudo-labels and consistency regularization to leverage unlabeled data, it effectively utilizes diverse evidence from different perspectives, thereby reducing the risk of mispredictions. The Dempster-Shafer theory is used to integrate supplementary evidence from the same sample to obtain more reliable overall evidence. With the support of fused evidence, more comprehensive and accurate pseudo-labels can be generated. By adding unlabeled samples and their pseudo-labels to the training set, the model can achieve excellent performance even with insufficient labeled data.

[0034] Example 1

[0035] A semi-supervised image classification method based on evidence theory, comprising the following steps:

[0036] 101: The image dataset used for semi-supervised learning is processed by randomly selecting the same number of small samples from each category and retaining the labels of the small number of samples to form the labeled dataset used for training; the remaining unselected samples form the unlabeled dataset used for training, that is, the label information of this part of the samples is not used during training.

[0037] The preprocessing process in step 101 mainly includes the following steps:

[0038] 1011: Randomly select the same number of samples from each category from the original dataset, retain their labels, and use the selected samples and their labels as the labeled dataset;

[0039] 1012: The labels of the remaining unsampled samples are discarded, and the samples are kept separately as an unlabeled dataset;

[0040] 1013: Labeled and unlabeled datasets together constitute the dataset used to train the model.

[0041] 102: Adding varying degrees of perturbation to the original image using data augmentation techniques;

[0042] Data augmentation methods are categorized into two types based on the strength of the added perturbation: weak augmentation (W()) and strong augmentation (S()). Weak augmentation involves random horizontal flipping, cropping, and rotation of the original image; strong augmentation utilizes RandAugment technology to randomly augment the original image, which adaptively selects the most suitable augmentation strategy.

[0043] The data augmentation techniques in step 102 mainly include the following steps:

[0044] 1021: In weak enhancement, the probability of randomly flipping the image is 0.5, and the image is cropped into squares of the same size. The padding of the image border is 0.125 times the side length of the image, and the padding type is reflection.

[0045] 1022: The RandAugment technique used in strong augmentation is an automatic data augmentation method. RandAugment defines an operation set including 14 operations: Identity, AutoContrast, Equalize, Rotate, Solarize, Color, Posterize, Contrast, Brightness, Sharpness, ShearX, ShearY, TranslateX, and TranslateY. RandAugment includes two positive integer parameters: N and M, where N represents the number of operations used in each augmentation, and these N operations are drawn from the operation set with equal probability; M represents the magnitude of all operations when applied. In this embodiment of the invention, the values ​​of N and M are set to 2 and 10, respectively.

[0046] 103: Perform weak augmentation on labeled samples, and train the model using the weakly augmented samples in a supervised learning manner.

[0047] The specific process of training the network using labeled samples is as follows:

[0048] 1031: For N l Weak augmentation is performed on each labeled sample x, and the augmented sample is fed into each network branch f. k () to obtain the output prediction probability f k (W(x));

[0049] 1032: Obtaining classification probabilities using the softmax activation function

[0050] 1033: The supervised loss is calculated by multiplying the predicted probability of each network branch for the augmented labeled sample by the cross-entropy of the sample label y.

[0051] in, For network f k () is the predicted probability distribution of the sample.

[0052] 104: Weak and strong augmentation are performed on unlabeled samples respectively. The weakly augmented samples are used to generate and retain pseudo-labels, while the strongly augmented samples are used to enhance the classification accuracy and generalization of the model.

[0053] 105: Weakly enhanced unlabeled samples are fed into each subnetwork to generate evidence. The uncertainty of each subnetwork can be obtained by modeling through Dirichlet distribution. The evidence and uncertainty obtained by fusing evidence theory are used to obtain fused evidence. Based on the fused evidence, more reliable classification results and pseudo-labels can be obtained.

[0054] The specific process for generating and selecting pseudo-labels for unlabeled samples is as follows:

[0055] 1051: First, divide the N into M categories. u x unlabeled samples u Weak enhancement is performed to obtain the enhanced sample W(x) u );

[0056] 1052: The enhanced sample W(x) u The data is fed into each branch network to obtain different output evidence, i.e. For network f k () Evidence information belonging to category M obtained;

[0057] 1053: Obtaining the parameters of the Dirichlet distribution from the output evidence of the branch network. For network f k () The parameters of the Dirichlet distribution obtained in the M-classification task.

[0058] 1054: The Dirichlet normalization factor is obtained from the parameters of the Dirichlet distribution.

[0059] in, For network f k () The parameters of the Dirichlet distribution obtained in the M-classification task; For network f k () The evidence information obtained belongs to the M category.

[0060] 1055: The belief mass for each class can be obtained using the Dirichlet normalization factor and the parameters of the Dirichlet distribution. Uncertainty in branch network prediction and u k The larger the value, the greater the uncertainty of the evidence output by the network;

[0061] 1056: Using evidence fusion theory to fuse all branch networks and u k To obtain more reliable b and u, the fusion process is as follows when k=2:

[0062]

[0063]

[0064] in, These are parameters used for normalization. When k ≥ 3, they are merged pairwise in a merge manner. The information belonging to the m-th class extracted for the first branch network. The information belonging to the nth class is extracted for the second branch network.

[0065] 1057: α for classification prediction can be obtained from the fused b and u, and the calculation process is as follows:

[0066]

[0067] e = [b1×S,b2×S,...,b M ×S],

[0068] α=[e1+1,e2+1,...,e M +1];

[0069] Among them, b M For the merged information belonging to class M, e M The evidence belonging to class M is derived from the fused information, where M is the total number of classes in the classification task, S is the Dirichlet normalization factor, and α is the parameter of the Dirichlet distribution.

[0070] 1058: The classification probability p = softmax(α) is obtained by applying the softmax activation function to the obtained α, where p is the prediction result of integrating all branch networks;

[0071] 1059: The pseudo-label of the one-hot encoding can be obtained from the classification probability p, i.e.

[0072] 106: The specific process of training the network using unlabeled samples and their pseudo-labels is as follows:

[0073] 1061: N of M categories u x unlabeled samples u Sample S(x) after k different enhancements u The data is fed into a k-branch network and processed through a softmax activation function to obtain classification predictions.

[0074] 1062: The unsupervised loss is calculated by taking the cross-entropy between samples whose classification probability is greater than a threshold and their pseudo-labels. Where I() is an indicator function, that is, when max(p)≥τ is true, the output value of I(max(p)≥τ) is 1, otherwise it is 0.

[0075] 1063: The total training loss for each branch network is obtained by weighting the supervised and unsupervised losses. Where λ is a parameter that weighs the contribution of labeled and unlabeled samples.

[0076] 1064: When performing image classification tasks, data is fed into each trained network. The network extracts information from the training data and fuses the extracted information using evidence theory to generate fused evidence information.

[0077] 1065: The fused evidence information is used to make predictions through the softmax activation function. The category with the highest predicted probability value is used as the final output category. The final output category is used for image classification to improve the classification accuracy of the image.

[0078] Example 2

[0079] The feasibility of the scheme in Example 1 is verified on a specific dataset below, as detailed in the following description:

[0080] The experimental details on the CIFAR-10 and SVHN datasets are as follows:

[0081] 1. Experimental environment:

[0082] The hardware configuration consists of 4 Titan X Pascal GPUs, the programming language and version is Python 3.8, the open source environment and version is Anaconda3, the deep learning framework is PyTorch, and the versions of torch and torchvision are 1.4 and 0.5 respectively.

[0083] 2. Experimental Background:

[0084] In image classification applications, labeling data is resource-intensive, so the available datasets often consist of a portion of labeled data and a large amount of unlabeled data. Semi-supervised image classification uses such datasets to train models, aiming to fully mine and utilize the information contained in the unlabeled data with the help of a small amount of labeled data. This allows the model to learn better semantic information and improve classification performance.

[0085] 3. Data source:

[0086] The CIFAR-10 and SVHN datasets used in the experiments are public datasets. The deep learning framework PyTorch provides interfaces for these two datasets, which can be loaded using the `torch.utils.data.Dataset` class. The CIFAR-10 dataset consists of 32×32 RGB color images, with 10 categories: airplane, car, bird, cat, deer, dog, frog, horse, boat, and truck, each with 6000 images. The SVHN dataset also consists of 32×32 RGB color images, with numbers 1-9 labeled 1-9, and "0" labeled 10. The SVHN training set contains 73257 images, and the test set contains 26032 images.

[0087] 4. Experimental Procedure: First, acquire the necessary hardware computing resources and software development environment, including GPU and Anaconda. Then, develop the code using the PyTorch deep learning framework, including building the network structure, reading training data, preprocessing the data, and setting training parameters. Finally, evaluate the trained model using a test set and compare it with other methods.

[0088] 5. Experimental Results:

[0089] The evaluation results of the present invention and the comparison method on the CIFAR-10 and SVHN datasets are shown in Table 1 below. The data presented in Table 1 represent the classification error rate, where "labels" indicates the number of labeled data points in the dataset. The experimental results show that the semi-monitored model (EPLE) of the present invention has a lower classification error rate than the comparison method, meaning that EPLE has higher classification accuracy.

[0090] Table 1

[0091]

[0092] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A semi-supervised image classification method based on evidence theory, characterized in that, The method comprises: Processing the image data set used by the semi-supervised learning, randomly drawing the same number of small samples from each category respectively, and retaining the labels of the small samples to form a labeled data set used in training; Adding different degrees of disturbance to the original image through a data enhancement technique; weakly enhancing the labeled samples, and training the model in a supervised learning manner by using the samples after weak enhancement; Respectively performing weak enhancement and strong enhancement on the unlabeled samples, feeding the weakly enhanced version of the unlabeled samples into each subnetwork to generate evidence, obtaining the uncertainty of each subnetwork through Dirichlet distribution modeling, obtaining the fused evidence by fusing the obtained evidence and uncertainty by using evidence theory, and obtaining more reliable classification results and pseudo labels based on the fused evidence; Training the network by using the unlabeled samples and the pseudo labels, and classifying the images by using the network after continuous training.

2. The semi-supervised image classification method based on evidence theory according to claim 1, characterized in that, The specific process of feeding the weakly enhanced version of the unlabeled samples into each subnetwork to generate evidence, obtaining the uncertainty of each subnetwork through Dirichlet distribution modeling, obtaining the fused evidence by fusing the obtained evidence and uncertainty by using evidence theory, and obtaining more reliable classification results and pseudo labels based on the fused evidence is as follows: Feeding the weakly enhanced samples into each branch network to obtain different output evidences; obtaining the parameters of the Dirichlet distribution from the output evidence of the branch network; obtaining a Dirichlet normalization factor from the parameters of the Dirichlet distribution; and obtaining the belief mass of each class by the Dirichlet normalization factor and the parameters of the Dirichlet distribution and the uncertainty of the branch network prediction and The evidence fusion theory is used to fuse all branch networks and u k When k = 2, the fusion process is as follows: wherein, is a parameter for normalization, when k≥3, merging in pairs, is information belonging to the m-th class extracted by the first branch network, is information belonging to the n-th class extracted by the second branch network; Obtaining α used for classification prediction by fusing b and u, and the calculation process is as follows: e = [b1x S, b2x S,..., b M x S], a = [el + 1, e2 + 1,..., e M + 1]; where b M is the fused information belonging to the m-th class, e M is the evidence converted from the fused information belonging to the m-th class, M is the total number of classes of the classification task, S is a Dirichlet normalization factor, and a is a parameter of the Dirichlet distribution; a classification probability is obtained by subjecting the obtained a to a softmax activation function; and a pseudo label encoded by a one-hot code is obtained from the classification probability when the prediction probability is higher than a threshold value τ.

3. The semi-supervised image classification method based on evidence theory according to claim 1, characterized in that, The method further comprises: Feeding the unlabeled samples with pseudo labels into each branch network after strong enhancement, performing a strong enhancement operation on the samples before feeding them into each network; obtaining classification probabilities by performing a softmax activation function on the outputs of each branch network; and calculating the cross-entropy between the classification probabilities and the pseudo labels as an unsupervised loss for network parameter updating.

Images