A collaborative learning method for image classification with noisy labels

Abstract

The present invention relates to the field of deep learning technology, and in particular to a collaborative learning method for classifying images with noisy labels. While retaining the collaborative training of the Co-teaching method, the present invention makes two improvements. First, the cross-training method is abandoned, and a one-way data exchange method is adopted. Second, the loss function of the data receiver network is modified, so that the loss functions of the data receiver and the data provider are different. Combined with a symmetric cross-entropy loss function and label smoothing regularization, the present invention suppresses the influence of noisy labels on image classification through small-loss training with one-way data exchange, improves robustness through the symmetric cross-entropy loss function, and prevents overfitting through label smoothing regularization.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a collaborative learning method for classifying images with noisy labels. Background Art

[0002] The current mainstream image classification methods with noisy labels are methods based on robust loss functions, methods based on robust probability frameworks, and methods based on denoising.

[0003] There are many conditions for the robust loss function method. For example, the symmetry condition must be met to be robust to uniform noise. Therefore, it has great limitations in dealing with the problem of image classification with noisy labels. On the other hand, it is overfitting. In the classification of images with noisy labels, the network will first learn easy samples and then learn difficult samples. In order to learn all the sample data, excessive training rounds are triggered. Summary of the Invention

[0004] The purpose of the present invention is to provide a collaborative learning method for classifying images with noisy labels, aiming to solve two technical problems encountered in the problem of image classification with noisy labels: one is that the conditional dependence is too large, and many robust loss functions have many theoretical premises; the other is the serious overfitting problem.

[0005] To achieve the above object, the present invention provides a collaborative learning method for classifying noisy labeled images, comprising the following steps:

[0006] Step 1: Input small batches of data into network A and network B for training respectively to obtain the training samples selected by network A;

[0007] Step 2: The training sample is used for the forward propagation of the next min-batch of network A, and is also passed to network B for the forward propagation of the next min-batch of network B.

[0008] Step 3: In the next min-batch, network A and network B respectively receive the data selected in the previous min-batch and continue to propagate and calculate the loss, repeating steps 1 and 2 until the classification model of network B is output.

[0009] Furthermore, in step 1, network A is the data provider and network B is the data receiver. After receiving the data, network A extracts features, calculates the loss, uses the loss value to select samples, and selects training samples based on the loss value sorting.

[0010] Furthermore, in step 2, the loss function used by the A network is the cross entropy loss function.

[0011] Furthermore, in step 2, the B network adopts two loss functions, namely the cross entropy loss function and label smoothing regularization.

[0012] Furthermore, the labels in the label smoothing regularization adopt one-hot vectors.

[0013] The present invention provides a collaborative learning method for classifying images with noisy labels. While retaining the collaborative training of the co-teaching method, two improvements are made. First, the cross-training method is abandoned and a one-way data exchange method is adopted. Second, the loss function of the data receiver network is modified. The loss functions of the data receiver and the data provider are different. Combined with the symmetric cross-entropy loss function and label smoothing regularization, the present invention suppresses the influence of noisy labels on image classification through small-loss training with one-way data exchange, improves robustness through the symmetric cross-entropy loss function, and prevents overfitting through label smoothing regularization. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 This is a flowchart of a collaborative learning method for classifying noisy labeled images according to the present invention.

[0016] Figure 2 This is an example diagram comparing the data flow and co-teaching of a collaborative learning method for noisy label image classification of the present invention.

[0017] Figure 3 FIG. 1 is a schematic diagram of the Co-SCE test process in a CIFAR-10 dataset with 50% uniform noise according to a specific embodiment of the present invention.

[0018] Figure 4 FIG. 1 is a schematic diagram of the Co-SCE testing process in a specific embodiment of the present invention using 45% non-uniform noise in the CIFAR-10 dataset.

[0019] Figure 5 2 is a comparison diagram of the Co-SCE and Co-LSR of the specific embodiment of the present invention in the 20% non-uniform noise-test process of the CIFAR-100 dataset. DETAILED DESCRIPTION

[0020] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0021] Related background technologies and terms in this invention include collaborative training algorithm, Symmetric Cross Entropy (SCE) loss function, Label Smoothing Regularization (LSR)

[0022] In traditional supervised learning image classification methods, classifiers typically learn from a large amount of labeled sample data, then build models to predict unseen sample data. However, in many application scenarios, obtaining large amounts of labeled data is often difficult, as obtaining these labeled samples can be labor-intensive and resource-intensive. If learning is performed using a small number of labeled samples, the resulting model often lacks generalization performance. Therefore, learning based on a small number of labeled samples has become an important area of machine learning, with collaborative learning algorithms such as co-training being a prominent example. The key idea behind this algorithm is to cross-train two classifiers on a small number of labeled datasets, then learn from a large number of unlabeled samples, and then incorporate these learned samples into the classifiers for continued training.

[0023] The SCE loss function is a robust loss-based solution for image classification with noisy labels. It combines the robust inverse cross entropy loss function with the cross entropy loss function. On the one hand, it takes advantage of the cross entropy loss function's fast training speed and easy convergence; on the other hand, it takes into account the inverse cross entropy loss function's robustness to noisy label samples.

[0024] Label smoothing regularization is a method for preventing overfitting. Its application requires two conditions: first, the classification labels are one-hot vectors, and second, the loss function must be a cross-entropy loss function. The main consideration is that if the model learns to assign partial probability to the true value label for each training instance, it will not be able to generalize effectively. In other words, the model will continuously increase the gap between the true and false labels. This will cause the final output correct labels to be close to 1 and the false labels to be close to 0, which greatly increases the risk of overfitting. Therefore, the original one-hot labels are modified by adding a label smoothing term. This makes the final label lower bound non-zero, reduces the distance between the maximum logit and all other logits, and prevents overfitting.

[0025] See also Figure 1, the present invention proposes a collaborative learning method for classifying images with noisy labels, comprising the following steps:

[0026] S1: Input small batches of data into network A and network B for training respectively to obtain the training samples selected by network A;

[0027] S2: The training sample is used for the forward propagation of the next min-batch of network A, and is also passed to network B for the forward propagation of the next min-batch of network B;

[0028] S3: In the next min-batch, network A and network B respectively receive the data selected in the previous min-batch and continue to propagate and calculate the loss, repeating steps S1 and S2 until the classification model of network B is output.

[0029] In step S1, network A is the data provider and network B is the data receiver. After receiving the data, network A extracts features, calculates the loss, uses the loss value to select samples, and selects training samples based on the loss value sorting.

[0030] In step S2, the loss function used by the A network is the cross entropy loss function.

[0031] In step S2, the B network adopts two loss functions, namely cross entropy loss function and label smoothing regularization.

[0032] Specifically, the labels in the label smoothing regularization adopt one-hot vectors.

[0033] The following is a further explanation based on the specific steps:

[0034] Step S1: Input small batch data to network A and network B, such as Figure 2 As shown in Figure 1, two network models are used for training, where A is the data provider and B is the data receiver. After network A receives the data, it extracts features and then calculates the loss. The loss value is used to select samples, where the sample selection strategy is shown in formula (1). The specific meaning of R(t) is the proportion of sample selection in the tth round of training. Since the network always learns the noise-free label samples first and then the noise-labeled samples, it will not overfit at the beginning and all samples can be used for learning. As the learning progresses, the noise labels will be gradually learned, and the loss value of the noise label samples will be larger. Then the losses of all samples are sorted and the first R(t) samples are selected as training samples.

[0035]

[0036] Where T k, c, τ are hyperparameters. If the noise rate η is known, τ = η; if it is unknown, it can be selected by cross-validation. k How many rounds does it take for the selection rate to reach the minimum value? c represents the decreasing change of the selection rate. When c = 1, it shows a linear decreasing change. k In each round, the sample selection rate R(t) decreases.

[0037] Step S2: The samples selected by network A are used for the forward propagation of its own next min-batch on the one hand, and are passed to network B on the other hand for the forward propagation of the next min-batch of network B. Min-batch refers to the small batch of data input to the network each time. The loss function of network A is the cross-entropy loss function, which mainly uses the non-robustness of the cross-entropy loss function to select samples. Network B adopts two loss functions in the present invention, one is SCE and the other is LSR. Network B adopts SCE and LSR mainly because they are robust to noisy labels and have the ability to resist overfitting. SCE is a symmetric cross entropy, a robust loss function, and its specific form is shown in formula (2); LSR is a label smoothing regularization, which has the ability to resist overfitting, and its specific form is shown in formula (3).

[0038]

[0039] Among them, α and β are two hyperparameters of the cross entropy loss function and the inverse cross loss function respectively. q(k|x) represents the value of the sample x label in the kth dimension. The label is generally a one-hot vector. p(k|x) is the probability of sample x output in the kth dimension after the network model.

[0040]

[0041] Where ∈ is the smoothing parameter. q' is the corrected label, p is the probability of the network output, and K represents the number of sample categories.

[0042] Step S3: In the next mini-batch, networks A and B receive the data selected in the previous mini-batch on network A, continue forward propagation, calculate the loss, and then repeat steps 1 and 2. Finally, the classification model of network B is output.

[0043] Specifically, both networks A and B receive data from the previous mini-batch selected by network A. Networks A and B continue forward propagation, calculating the loss. The loss value is then used to update the parameters via backward propagation. Network A then uses the updated parameter model to select a new mini-batch of samples and passes it to network B, repeating steps 1 and 2. Training completes, and the classification model for network B is output.

[0044] Furthermore, the present invention also proposes a specific embodiment, which is experimentally verified on three public datasets: MNIST, CIFAR-10, and CIFAR-100:

[0045] MNIST is a dataset of grayscale handwritten digit images. CIFAR-10 is a small color dataset for recognizing common objects, containing 10 categories: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. CIFAR-100 is a large color dataset for common objects, divided into 20 superclasses, each with 5 subclasses, for a total of 100 categories. Two main types of noise labels are used: symmetric noise (uniform noise), which is achieved by uniformly flipping the labels of a given proportion of training examples to other class labels; and asymmetric noise (class-correlated noise), which flips a given class to a specified class at a certain ratio. The specific settings for asymmetric noise are as follows: in the MNIST dataset, 7 is flipped to 2, 2 is flipped to 7, 3 is flipped to 8, 5 is flipped to 6, and 6 is flipped to 5. In the CIFAR-10 dataset, truck is flipped to automobile, bird is flipped to airplane, cat is flipped to dog, dog is flipped to cat, and deer is flipped to horse. In the CIFAR-100 dataset, superclasses are flipped pairwise.

[0046] See also Figures 3 to 5 In this embodiment, the three data sets are trained using the same network structure, which is composed of 9 convolutional layers, 2 pooling layers, and 1 fully connected layer. The specific network design and its related parameters are shown in Table 1.

[0047] Table 1 Network structure and related network parameter settings of the present invention

[0048]

[0049] The datasets also target two types of noise: symmetrical noise and asymmetrical noise. For symmetrical noise, the noise rates are set at 20% and 50% across all three datasets; for asymmetrical noise, the noise rates are set at 20% and 45% across all three datasets.

[0050] The structure of directly using a single network for sample selection is called Small-loss. The main consideration of sample selection is to select samples with "small loss". Within the parameter selection range, select the best T kThe experimental results of Co-SCE, Small-loss and SCE on three datasets are shown in Table 2.

[0051] Table 2 Comparison results of Co-SCE, Small-loss and SCE experiments

[0052]

[0053]

[0054] Each cell contains two data points: the maximum value of the current experimental test set and the average of the last 10 epochs. The average of the last 10 epochs is used to reflect the model's ability to resist overfitting. Table 2 shows that Co-SCE outperforms SCE and Small-loss on all three datasets, regardless of whether the noise is uniform or non-uniform. Specifically, SCE suffers from severe overfitting when the relative noise rate is high, while Co-SCE is generally more resistant to overfitting. This is primarily due to its sample selection strategy, which results in relatively less noisy data in the late stages of training. Co-SCE and Co-LSR were also compared experimentally using non-uniform noise on CIFAR-10 and CIFAR-100, as shown in Table 3. The experimental results show that Co-LSR significantly outperforms Co-SCE on the test set classification performance on both CIFAR-10 and CIFAR-100 datasets with non-uniform noise.

[0055] Table 3 Experimental results of Co-SCE and Co-LSR on non-uniform noise

[0056]

[0057] Table 4 is a comparison of Co-SCE and Co-teaching in uniform noise experiments. For the simple MNIST dataset, the Co-SCE method is better than Co-teaching. It is also noted that the average test value of Co-SCE in the last 10 epochs is significantly better than Co-teaching. In the CIFAR-10 dataset, the two methods have comparable results. In the CIFAR-100 dataset, the Co-SCE proposed in this invention is slightly inferior to Co-teaching in the classification accuracy of the test set. There are two main reasons for this. One is that the sample selection strategy for complex datasets has limited ability to filter noisy label samples, and the other is that the SCE loss function has limited processing capabilities for complex datasets. In general, the Co-SCE method and Co-teaching each have their own advantages in datasets of different complexity, and their overall performance is comparable.

[0058] Table 4 Comparison of uniform noise experiments between Co-SCE and Co-teaching

[0059]

[0060] The above disclosure is only a preferred embodiment of the present invention, and certainly cannot be used to limit the scope of the rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A collaborative learning method for classifying noisy labeled images, characterized in that: The following steps are involved: Step 1: Input small batches of data into network A and network B for training respectively to obtain the training samples selected by network A; In step 1, network A is the data provider and network B is the data receiver. After receiving the data, network A extracts features, calculates the loss, uses the loss value to select samples, and selects training samples based on the loss value sorting. Step 2: The training sample is used for the forward propagation of the next min-batch of network A, and is also passed to network B for the forward propagation of the next min-batch of network B. In step 2, the loss function used by network A is the cross entropy loss function; network B uses two loss functions, namely the cross entropy loss function and label smoothing regularization; Step 3: In the next min-batch, network A and network B respectively receive the data selected in the previous min-batch and continue to propagate and calculate the loss, repeating steps 1 and 2 until the classification model of network B is output.

2. The collaborative learning method for noisy label image classification according to claim 1, wherein: The labels in the label smoothing regularization adopt one-hot vectors.

Images