A robust denoising framework for image classification with noisy labels

Abstract

The present invention relates to the field of deep learning technology, and in particular to a noisy labeled image classification method in a robust denoising framework. First, original data is input into a pre-trained convolutional network for forward propagation. The predicted label and the true label are compared using the SCE loss function to calculate the loss. Backpropagation is then performed to update the parameters, and the pre-trained model is output. Then, the O2U-Net training method is used to remove noisy labels as much as possible. Finally, the pre-trained model is loaded with the denoised data, and the loss is calculated using the SCE loss function. The loss is backpropagated to update the network parameters until the end of the training round, and the final classification model is output. The present invention uses a noise label cleaning scheme to reduce the noise rate, and then uses a robust loss function under low noise rate conditions to achieve better classification accuracy than using the robust loss function or the noise label cleaning scheme alone.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a noisy labeled image classification method in a robust denoising framework. Background Art

[0002] For image classification with noisy labels, the current mainstream approaches are those based on robust loss functions and denoising. Robust loss methods rely on the type of noise, for example, requiring uniform noise. In reality, datasets contain diverse noise labels, which do not meet the theoretical requirements for robust loss functions. Denoising methods based on denoising often face the risk of inadequate denoising and false cleanups. This is especially true for datasets with high noise levels. Denoising significantly reduces the amount of data after noise removal, and if noise remains, classification performance will deteriorate. Summary of the Invention

[0003] The purpose of the present invention is to provide a noisy label image classification method in a robust denoising framework, which uses a noise label cleaning scheme to reduce the noise rate, and then uses a robust loss function under low noise rate conditions to obtain better classification accuracy than using the robust loss function or the noise label cleaning scheme alone.

[0004] To achieve the above objectives, the present invention provides a method for classifying noisy labeled images in a robust denoising framework, comprising the following steps:

[0005] The original data is input into the pre-trained convolutional network and processed by the SCE loss function to obtain the pre-trained model;

[0006] Remove noisy labels from the original data to obtain denoised samples;

[0007] The pre-trained model is loaded, and the denoised samples are inputted at the same time, and the training output is the final classification model.

[0008] Among them, the process of obtaining the pre-trained model is specifically as follows: first, the original data is input into the pre-trained convolutional network, the prediction scores of each category are obtained through feature extraction, and then the loss is calculated using the SCE loss function, back propagation is performed, and the network parameters are updated to finally obtain the pre-trained model.

[0009] Among them, the SCE loss function includes the cross entropy loss function and the inverse cross entropy loss function. The cross entropy loss function is responsible for maintaining the speed and generalization of training, and the inverse cross entropy is robust to noisy labels and can reduce the degree of overfitting of the problem.

[0010] In the process of removing noisy labels from the original data to obtain denoised samples, the O2U-Net training method is used as the denoising framework.

[0011] Among them, the process of training and outputting the final classification model is specifically to load the pre-trained model, input the sample at the same time, use the secondary training convolutional network to extract features of the sample, and then use the SCE loss function to calculate the loss, and backpropagate the loss to update the network parameters until the training round is completed and the final classification model is output.

[0012] The present invention provides a noisy label image classification method in a robust denoising framework. First, the original data is input into a pre-trained convolutional network for forward propagation. The predicted label and the true label are calculated using the SCE loss function. The parameters are updated through backpropagation, and the pre-trained model is output. Then, the O2U-Net training method is used to remove noisy labels as much as possible. Finally, the pre-trained model is loaded with the denoised data, and the loss is calculated using the SCE loss function. The loss is backpropagated to update the network parameters until the end of the training round, and the final classification model is output. The present invention uses a noise label cleaning scheme to reduce the noise rate, and then uses a robust loss function under the condition of low noise rate to obtain better classification accuracy than using the robust loss function or the noise label cleaning scheme alone. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1 It is a flowchart of a noisy labeled image classification method of a robust denoising framework of the present invention.

[0015] Figure 2 This is a schematic diagram of the idea structure of a noisy label image classification method in a robust denoising framework of the present invention.

[0016] Figure 3 This is a comparison diagram of 40% uniform noise in the CIFAR-10 dataset, a specific embodiment of the present invention.

[0017] Figure 4 This is a comparison diagram of 60% uniform noise in the CIFAR-10 dataset, a specific embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0019] The relevant background technologies and terms in the present invention include O2U-Net (Overfitting To Underfitting) network framework and Symmetric Cross Entropy (SCE) loss function.

[0020] The O2U-Net framework is a network framework for classifying noisy labeled images based on removing noisy labels. It primarily adjusts the learning rate during training to keep the model oscillating between underfitting and overfitting. The loss of all samples is then recorded throughout the training process. Using this principle, during underfitting, non-noisy labeled samples are fitted first, resulting in a low loss, while noisy labeled samples remain underfitted and have a high loss. During overfitting, the loss of both types of samples is relatively low. By adjusting the learning rate, the model oscillates between underfitting and overfitting, ultimately resulting in a high loss for noisy labeled samples and a low loss for non-noisy samples. This characteristic allows the model to filter out noisy labeled samples.

[0021] The SCE loss function is a solution for noisy label image classification based on a robust loss function, mainly by combining the robust inverse cross entropy loss function with the cross entropy loss function.

[0022] See also Figure 1 , the present invention proposes a noisy label image classification method based on a robust denoising framework, comprising the following steps:

[0023] S1: The original data is input into the pre-trained convolutional network and processed by the SCE loss function to obtain the pre-trained model;

[0024] S2: removing noisy labels from the original data to obtain denoised samples;

[0025] S3: Load the pre-trained model, input the denoised samples, and train and output the final classification model.

[0026] The process of obtaining the pre-trained model is as follows: first, the original data is input into the pre-trained convolutional network, the prediction scores of each category are obtained through feature extraction, and then the loss is calculated using the SCE loss function, back propagation is performed, and the network parameters are updated to finally obtain the pre-trained model.

[0027] The SCE loss function includes the cross entropy loss function and the inverse cross entropy loss function. The cross entropy loss function is responsible for maintaining the speed and generalization of training, and the inverse cross entropy is robust to noisy labels and can reduce the degree of overfitting of the problem.

[0028] In the process of removing noisy labels from the original data to obtain denoised samples, the O2U-Net training method is used as the denoising framework.

[0029] The process of training and outputting the final classification model is specifically to load the pre-trained model, input the sample at the same time, use the secondary training convolutional network to extract features of the sample, and then use SCE to calculate the loss, backpropagate the loss to update the network parameters until the training round is completed and the final classification model is output.

[0030] The following is a further explanation based on the specific execution steps:

[0031] Step S1: First, the original data is input into the pre-trained convolutional network. Through feature extraction, the prediction scores of each category are obtained. Then, the loss is calculated using SCE, backpropagation is performed, and the network parameters are updated. Finally, the pre-trained model is obtained for use in step S3.

[0032] The SCE loss function is shown in formula (1):

[0033]

[0034] Here, α and β are the two hyperparameters of the cross-entropy loss function and the inverse cross-entropy loss function, respectively. q(k|x) represents the value of the label of sample x in the kth dimension. The label is generally a one-hot vector. p(k|x) represents the probability of sample x output in the kth dimension after passing through the network model. Directional propagation is performed using the SCE robust loss function, which mainly consists of two parts: the cross-entropy loss function and the inverse cross-entropy loss function. The cross-entropy loss function is primarily responsible for maintaining training speed and generalization, while the inverse cross-entropy loss function is robust to noisy labels and can reduce the degree of overfitting. Finally, a more robust pre-trained model is obtained through step one.

[0035] Step S2: Using the cross-entropy loss function, we periodically adjust the learning rate to remove noise labels. We mainly use the O2U-Net method to remove noise labels. We set two learning rates r1 and r2, where r1>r2, and use a linear increasing function to change the learning rate cyclically. The following is the learning rate adjustment equation during the cyclic training process:

[0036]

[0037] r(t)=(1-s(t))×r1+s(t)×r2 (3)

[0038] Where t represents the number of training rounds, c represents the total number of rounds per cycle, s(t) represents the learning rate distribution ratio, and mod is the modulus operation, which calculates the remainder. r(t) is the learning rate for the tth round. As the noise rate decreases from high to low, the noisy labels will gradually tend to overfit, and the loss value will decrease from large to small, while the loss value of the noise-free label samples will remain relatively unchanged. Throughout the process, the loss value of the noisy labels will be larger. The loss values ​​are then sorted to eliminate the label samples with larger loss values. The cross-entropy loss function is shown in Formula (4).

[0039]

[0040] Where q(k|x) represents the value of the sample x label in the kth dimension. The label is generally a one-hot vector, and p(k|x) is the probability of sample x output in the kth dimension after the network model.

[0041] Step S3: First, load the pre-trained model parameters from step S1 and simultaneously input the sample denoised in step S2. The sample is then subjected to feature extraction using the secondary training convolutional network. The loss is then calculated using SCE (Scaling Convolutional Efficiency) and backpropagated to update the network parameters until the training round is complete. Finally, the classification model is output. The loss function in this step is the same as that in step S1, primarily to account for the presence of unclean samples after step S2. The SCE loss function can be used to mitigate the impact of some noisy samples on classification.

[0042] The overall framework of this invention is mainly based on the improvement of O2U-Net. In O2U-Net, the loss functions of steps 1 and 3 are both cross-entropy loss functions. This invention creatively modifies the loss functions of steps 1 and 3. It mainly starts from the shortcomings of SCE and O2U-Net:

[0043] (1) The robustness of SCE is often targeted at uniform noise, but not so good at non-uniform noise. Therefore, in theory, this loss function has limitations in its application environment. At the same time, experiments show that SCE also performs poorly under high noise conditions.

[0044] (2) Although O2U-Net itself has a good denoising capability, it still cannot completely remove the noise and there are also cases of mis-cleaning. Therefore, these uncleaned noises will still affect the classification.

[0045] Taking these two points into account, the improved O2U-Net method presented in this paper effectively overcomes the shortcomings of both SCE and O2U-Net. Firstly, O2U-Net removes noisy labels, creating a relatively uniform and low-noise environment, thus overcoming the shortcomings of SCE. Secondly, SCE processes the remaining uncleaned noisy label samples, reducing their impact on classification, thus overcoming the shortcomings of O2U-Net. Therefore, the present invention is theoretically a more robust and effective method for classifying noisy images than SCE and O2U-Net.

[0046] Furthermore, the present invention also provides a specific embodiment, and experiments are conducted on three public datasets: MNIST, CIFAR-10, and CIFAR-100. MNIST is a grayscale handwritten digital image dataset. CIFAR-10 is a small color dataset for identifying common objects, which contains a total of 10 categories: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. CIFAR-100 is a large color dataset for common objects, divided into 20 superclasses, each with 5 subclasses, for a total of 100 categories. The noise label type mainly adopts two types of noise labels: one is symmetric noise, that is, uniform noise, which is obtained by uniformly flipping the labels of a given proportion of training samples to other class labels; the other is asymmetric noise, that is, class-related noise, which flips a certain category to a specified category according to a certain proportion. The asymmetric noise settings are as follows: In the MNIST dataset, 7 is flipped to 2, 2 is flipped to 7, 3 is flipped to 8, 5 is flipped to 6, and 6 is flipped to 5. In the CIFAR-10 dataset, truck is flipped to automobile, bird is flipped to airplane, cat is flipped to dog, dog is flipped to cat, and deer is flipped to horse. In the CIFAR-100 dataset, superclasses are flipped pairwise.

[0047] In terms of network architecture, the basic network architecture settings vary due to the different distributions of different datasets. The MNIST dataset uses a three-layer convolutional neural network for ease of learning. CIFAR-10 uses an eight-layer network structure, consisting of six convolutional layers and two fully connected layers. CIFAR-100 uses a ResNet34 network structure.

[0048] For symmetric noise labels, experiments were conducted at noise rates of 10%, 20%, 40%, 60%, and 80%. For asymmetric noise labels, experiments were conducted at noise rates of 10% and 20%. The experimental results are shown in the table below. It can be seen that on all three datasets, regardless of whether the noise is uniform or non-uniform, the proposed method achieves the highest classification accuracy on the test set. The specific experimental results are shown in Table 1.

[0049] Table 1 Experimental results

[0050]

[0051] At the same time, since the MNIST dataset is easy to learn, the differences between the several methods are not large, but the accuracy of the present invention is still slightly improved. In addition, the effect of not performing the preprocessing in step 1 on the MNIST dataset is slightly better. On the CIFAR-10 and CIFAR-100 datasets, it can be seen that the present invention has an improvement of 1 to 3 percentage points compared to SCE and O2U-Net, and the overall advantage is obvious. It can be seen that the higher the noise rate, the more obvious the overall advantage of the present invention on uniform noise labels. At the same time, the present invention also pays attention to the fact that compared with uniform noise, although the improvement of the present invention on non-uniform noise labels is smaller, it still has an advantage. Therefore, introducing a robust loss function into the denoising framework is effective for solving the problem of image classification with noisy labels.

[0052] See also Figure 3 and Figure 4 , Figure 3 and Figure 4 The comparison between 40% uniform noise and 60% uniform noise of CIFAR-10 and other methods on the test set is shown respectively.

[0053] The above disclosure is only a preferred embodiment of the present invention, and certainly cannot be used to limit the scope of the rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A method for classifying noisy labeled images in a robust denoising framework, characterized in that: The following steps are involved: The original data is input into the pre-trained convolutional network and processed by the SCE loss function to obtain the pre-trained model; The SCE loss function includes a cross entropy loss function and a reverse cross entropy loss function; The process of obtaining the pre-trained model is as follows: first, the original data is input into the pre-trained convolutional network, and the prediction scores of each category are obtained by feature extraction. Then, the loss is calculated using the SCE loss function, back propagation is performed, and the network parameters are updated to finally obtain the pre-trained model; Remove noisy labels from the original data to obtain denoised samples; In the process of removing noisy labels from the original data to obtain denoised samples, the O2U-Net training method is used as the denoising framework; Load the pre-trained model, input the denoised samples, and train the output of the final classification model; The process of training and outputting the final classification model is specifically to load the pre-trained model, input the sample at the same time, use the secondary training convolutional network to extract features of the sample, and then use the SCE loss function to calculate the loss, backpropagate the loss to update the network parameters until the training round is completed and the final classification model is output.

Images