Robust classification method for images with noise labels based on continuous learning time-varying regularization

Abstract

The invention discloses a robust classification method for images with noise labels based on continuous learning time-varying regularization, and the method comprises the steps: enabling the learning of the noise labels to be regarded as a continuous learning process which comprises a plurality of learning tasks, and enabling network parameters to be decomposed into a clean part and a noise part; the dynamic regularization technology is adopted to control the updating process of clean parameters and the updating process of noise parameters, and the method comprises the specific steps that conventional training is conducted on a data set with noise labels through cross entropy loss, the accuracy of a model in each epoch is tested through a verification data set, and a verification curve A is obtained; respectively carrying out normalization and reverse normalization processing on the verification curve to obtain an incremental curve A1 and a decline curve A2, and solving an incremental function alpha (t) and a decline function beta (t) corresponding to the curves by taking epochs as independent variables; and applying different dynamic regularization constraints to the clean parameter and the noise parameter by taking alpha (t) and beta (t) as dynamic coefficients. According to the robust classification method for the images with the noise labels, the combination of a continuous learning method and a noise label method is promoted, the setting of parameters is avoided, and the interference of the noise labels on the model is inhibited.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and efficient and reliable artificial intelligence, and in particular to a robust classification method for noisy labeled images based on continuous learning time-varying regularization. Background Technology

[0002] Label noise is prevalent in many high-end, high-precision real-world applications. In autonomous driving, models need to identify different targets in complex scenes, such as traffic signs, vehicles, and pedestrians, requiring a large amount of labeled data for training. However, labeling such large-scale, complex data is prone to errors. In remote sensing (RS) scene classification or retrieval tasks, noise (also known as label noise) is unavoidable in large-scale RS benchmark archives due to factors such as geographic location / registration errors, land cover changes, and the diversity of annotators' knowledge backgrounds. In hyperspectral image (HSI) analysis, label information is a scarce resource and is inevitably affected by both human and non-human factors, resulting in significant label noise. Although most supervised HSI classification methods have achieved good classification results, their performance drops sharply when training samples contain label noise. In medical diagnosis, incomplete patient descriptions of illness, incomplete testing, and the expert's lack of expertise or subjective bias can easily lead to misdiagnosis. Therefore, proposing robust learning methods for noisy labels is of profound significance for practical applications.

[0003] Currently, sample selection methods learn from simple samples at the beginning of training, but they tend to overlook some clean samples with significant losses. These samples are of great value to model training. Since data collection is already costly, discarding this useful information will result in a certain degree of waste. At the same time, reducing the sample size can also lead to a decrease in training performance. Regularization methods usually require setting additional hyperparameters and tuning these hyperparameters, which increases the complexity and time cost of model training. Summary of the Invention

[0004] To address the shortcomings of the existing technology, the present invention aims to provide a robust classification method for noisy labeled images based on continuous learning time-varying regularization. This method combines continuous learning with noisy labeling, while avoiding hyperparameter settings and suppressing the interference of noisy labels on the model. Thus, during the learning process of noisy labeled images, this method can maximize the use of previously learned clean parameters and use the noisy parameters to learn incremental differences that cannot be explained by the clean parameters.

[0005] The technical solution adopted by this invention to solve its technical problem is as follows:

[0006] A robust classification method for noisy labeled images based on continuous learning and time-varying regularization is presented. The noisy label learning is treated as a continuous learning process involving multiple learning tasks. The network parameters are decomposed into clean and noisy parts, and dynamic regularization is used to control the update process of the clean and noisy parameters respectively. The specific steps are as follows:

[0007] Cross-entropy loss is used to train the model on a noisy labeled dataset. The validation dataset is used to test the model's accuracy at each epoch, thus obtaining the validation curve A. The validation curve is then normalized and denormalized to obtain the increasing curve A1 and the decreasing curve A2. The increasing function α(t) and the decreasing function β(t) corresponding to the curves are obtained with the epoch as the independent variable.

[0008] By introducing parameter handling methods from catastrophic forgetting, the training parameters of the network are decomposed into clean parameters Θ. clean and noise parameter Θ noise Different dynamic regularization constraints are applied to the clean parameter and the noise parameter, respectively, using α(t) and β(t) as dynamic coefficients.

[0009] Furthermore, the target expression is:

[0010]

[0011] in, Let l represent the overall training objective loss function, and l be the cross-entropy loss function. It is the clean parameter of the previous epoch, and ||·|2 represents the Euclidean norm.

[0012] Furthermore, for clean parameters, update them correctly in the early stages of training, while restricting their updates in the later stages;

[0013] For noise parameters, limit their updates in the early stages and gradually increase them in the later stages.

[0014] Furthermore, the process of obtaining the increasing and decreasing functions is as follows:

[0015] In a given noisy labeled dataset Using the cross-entropy loss function The cross-entropy loss function is defined as follows for training the network:

[0016]

[0017] Where N is the number of samples and K is the number of classes. Indicates sample x i The corresponding real tags.

[0018] Furthermore, in the verification set Test the model and calculate the prediction accuracy A on the validation set. t :

[0019]

[0020] Where 1 is an indicator function, which takes the value 1 when the prediction is correct and 0 otherwise;

[0021] The accuracy curve was obtained:

[0022]

[0023] Where E is the total number of epochs;

[0024] Normalize the verification accuracy curve A to obtain the normalized curve A1:

[0025]

[0026] The verification accuracy curve is inversely normalized to obtain curve A2:

[0027] A′1=-A1

[0028]

[0029] The corresponding dynamic coefficient functions α(t) and β(t) are obtained through A1 and A2. The independent variable of the function is the epoch, and the dependent variable is the verification accuracy corresponding to each epoch.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] 1. The robust classification method for noisy labeled images based on continuous learning time-varying regularization exemplified in this invention treats noisy label learning as a continuous learning process, providing a new approach to noisy label learning. By controlling the learning process of different network parameters, it reduces the forgetting of correct memories and enhances the robustness of the network.

[0032] 2. The robust classification method for noisy labeled images based on continuous learning time-varying regularization exemplified in this invention can dynamically control the direction of parameter updates. The time-varying regularization coefficients are derived from the dynamic prediction of the network, avoiding the setting of hyperparameters and reflecting the influence of different parameters on training in real time. Therefore, this algorithm has stronger generalization ability and more practical application value. Attached Figure Description

[0033] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0034] Figure 1(a) shows the test accuracy curves under different noise rates, and (b) shows the parameter update diagrams on clean and noisy datasets.

[0035] Figure 2 This is a block diagram of a dynamically regularized network structure.

[0036] Figure 3 A schematic diagram illustrating the acquisition of time-varying coefficient functions;

[0037] Figure 4 These are rising and falling curves. Detailed Implementation

[0038] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] Dataset It contains K categories and N samples. It is a sample in a d-dimensional data space. Is with x i The corresponding real label, (x i ,y i The conditions for independent and simultaneous partitioning must be met. The task of classification is to obtain a... The mapping function f(·;Θ) makes the empirical risk of parameter Θ... minimize.

[0041]

[0042] Where l represents the loss function.

[0043] As datasets grow larger, various noises inevitably arise during the data collection process, disrupting the originally clean dataset. It was converted into a dataset with noisy labels. In This represents labels that may contain noise. If in the dataset... Performing standard classification training will yield a small batch at a certain time. The formula:

[0044]

[0045] Where b is The number of samples included.

[0046] exist Up along Minimize the direction pair Θ t If the parameters are updated, then:

[0047]

[0048] Where η is the learning rate.

[0049] As can be seen from formula (3), using the dataset During training, the network no longer has noise tolerance and is prone to overfitting to noisy labels, leading to reduced generalization performance. Therefore, minimizing the impact of noisy labels on training is of great significance.

[0050] The root cause of overfitting in deep neural networks is overparameterization of noisy labeled samples. For example... Figure 1 As shown in (a), after the network is trained to a certain extent, the network parameters overfit the noisy labels, leading to a decline in generalization performance. This application, starting from the memory characteristics of DNNs, views this decline as the network forgetting earlier correct memories during parameter updates. To combat the interference of noisy labels, this forgetting should be minimized. We found that the forgetting characteristics of learning with noisy labels are very similar to catastrophic forgetting in continuous learning. Both noisy label learning and continuous learning aim for the network to retain previous knowledge while learning new knowledge. Continuous learning research has found that, for catastrophic forgetting, the network parameters can be decomposed into task-shared parameters and task-specific parameters, with different parameters specifically targeting different aspects of the task. Inspired by this, we attempt to decompose the network parameters into clean and noisy parts in noisy label learning. Their updates are then differentially controlled during training. Figure 1 As shown in (b), since the influence of clean and noisy labels on parameter updates varies at different training stages, the control of clean and noisy parameters should also dynamically change with the training stage. This embodiment uses dynamic regularization technology to control the update process of clean and noisy parameters respectively, thereby achieving the purpose of combating noise label interference.

[0051] Dynamic regularization:

[0052] The structure diagram of the dynamic regularized network is as follows: Figure 2As shown, the model is first trained on a noisy labeled dataset using the cross-entropy loss function. A validation dataset is used to test the model's accuracy at each epoch, resulting in validation curve A. Normalization and denormalization of the validation curve yield increasing curve A1 and decreasing curve A2. Using the epoch as the independent variable, the increasing function α(t) and decreasing function β(t) corresponding to the curves are calculated. Then, by introducing parameter handling methods from catastrophic forgetting, the network's training parameters are decomposed into clean parameters Θ. clean and noise parameter Θ noise Different dynamic regularization constraints are applied to the clean parameters and noisy parameters using α(t) and β(t) as dynamic coefficients, respectively. For clean parameters, they are encouraged to update correctly in the early stages of training, while their updates are restricted in the later stages. For noisy parameters, their updates are restricted in the early stages and gradually increased in the later stages. The constraint on the clean parameters keeps them relatively static and converges to a more ideal position, rather than gradually deviating from the correct direction as noise knowledge increases. The constraint on the noisy parameters reduces the influence of noise labels while taking into account the learning of incremental differences, thus improving data utilization. In this way, at the beginning of each epoch of training, the previously learned clean parameters can be utilized to the maximum extent, and the incremental differences that cannot be explained by the clean parameters can be learned using the noisy parameters.

[0053] The target expression is:

[0054]

[0055] in, Let l represent the overall training objective loss function, and l be the cross-entropy loss function. It is the clean parameter of the previous epoch, and ||·|2 represents the Euclidean norm.

[0056] In traditional cross-entropy loss functions, parameter updates are uncontrolled. In the objective loss function, the second part cleanly controls the parameter Θ. clean The update direction is regularized according to distance using the Euclidean norm, which helps with parameter generalization. This is to ensure that clean parameters do not deviate from the previous parameters when updating. Too far back, forcing the model to learn knowledge in the current epoch that is closer to the representation learned in the previous epoch. The third part controls the noise parameter Θ. noise The update is used to control the absorption of noisy data, so that the neural network itself does not overfit the noise. α(t) and β(t) are obtained through the validation set accuracy curve function, which are time-varying coefficients used to constrain the regularization term, with the independent variable being the epoch.

[0057] The following are the steps for dynamic regularization.

[0058]

[0059]

[0060] Time-varying coefficient function:

[0061] By observing the process by which deep neural networks first memorize clean-label data and then gradually memorize noisy-label data, we can see that the memory of deep neural networks is dynamically changing. Therefore, the model's parameters Θ... clean and Θ noise The update constraints should also be dynamic. That is, in the early stages of training, the clean parameters Θ... clean The update speed should be relatively fast to encourage it to quickly learn from clean labeled data. Later, to avoid overfitting to incorrectly labeled data, the update speed should be reduced to prevent forgetting correct knowledge. (Noise parameter Θ) noise The update control should be consistent with the clean parameter Θ clean The control direction is opposite.

[0062] Based on the above analysis, the regularization coefficient should be dynamically changed during the design process, and α(t) should be a function that gradually increases with each epoch. This is because once α(t) becomes very large, This will decrease quickly, limiting the parameter changes between adjacent epochs to prevent them from changing too much, thus maintaining a relative stability of the clean parameter. Similarly, β(t) should be a function that decreases with each epoch to limit Θ. noise The update. There are many such functions; here, we choose to extract the dynamic coefficient functions α(t) and β(t) from the network's own training, such as... Figure 3 As shown. The advantages of this approach are twofold: first, α(t) and β(t) are derived from the network itself, better reflecting its characteristics; second, it avoids the need for manually designing functions and adjusting hyperparameters on different networks, as this method does not require setting any hyperparameters. The process of obtaining α(t) and β(t) is as follows:

[0063] In a given noisy labeled dataset Using the cross-entropy loss function The cross-entropy loss function is defined as follows for training the network:

[0064]

[0065] Where N is the number of samples and K is the number of classes. Indicates sample x i The corresponding real tags.

[0066] In the validation set Test the model and calculate the prediction accuracy A on the validation set. t :

[0067]

[0068] Where 1 is an indicator function, which takes the value 1 when the prediction is correct and 0 otherwise;

[0069] The accuracy curve was obtained:

[0070]

[0071] Where E is the total number of epochs;

[0072] Normalize the verification accuracy curve A to obtain the normalized curve A1:

[0073]

[0074] Invert and normalize the accuracy curve A to obtain curve A2:

[0075] A′1=-A1 (9)

[0076]

[0077] The corresponding dynamic coefficient functions α(t) and β(t) can be obtained from A1 and A2. The independent variable of the function is the epoch, and the dependent variable is the verification accuracy corresponding to each epoch.

[0078] On the CIFAR-10 dataset containing symmetric noise of 0.2 and 0.4, the function curves we obtained are as follows: Figure 4 As shown.

[0079] The following are the steps to obtain the time-varying coefficient functions α(t) and β(t).

[0080]

[0081]

[0082] Dataset:

[0083] The effectiveness of the proposed method was validated on three synthetic noise datasets (MNIST, CIFAR-10, and CIFAR-100) and two real noise datasets (CIFAR-10N and Food-101). These datasets are widely used in image classification tests and can be considered as benchmark datasets.

[0084] MNIST. This dataset contains 70,000 images of handwritten digits, each 28×28 pixels in size and grayscale. The images are divided into 10 classes, with approximately 7,000 images in each class. 6,000 images are used for training, and 1,000 images are used for testing. The entire dataset is divided into a training set of 60,000 images and a test set of 10,000 images.

[0085] CIFAR contains two datasets: CIFAR-10 and CIFAR-100. Both datasets contain 60,000 long color images, each 32×32 pixels in size. CIFAR-10 divides the images into 10 classes, with 6,000 images per class (5,000 for training and 1,000 for testing). CIFAR-100 divides the images into 100 classes, with 600 images per class (500 for training and 100 for testing). Each dataset contains a training set of 50,000 images and a test set of 10,000 images.

[0086] CIFAR-10N. Based on the CIFAR-10 dataset, each image was relabeled by three independent workers on a crowdsourcing platform, thus containing truly noisy labels. Depending on the final confirmation method for image labels, the dataset can be categorized into three types: aggregated, random, and worst-case. The aggregated method has a label noise rate of 9.03%, the worst-case method has a label noise rate of 40.21%, and the random method has three label noise rates: 17.23%, 18.12%, and 17.64%.

[0087] Food-101. This dataset contains 101 different food categories, totaling approximately 100,000 images, each 112×112 pixels in size. These images were collected from the internet and cover common food types such as fruits, vegetables, staple foods, and desserts. For each category, 250 clean, manually reviewed test images and 750 training images with real-world label noise are provided. Each category contains approximately 1000 high-quality images, with 750 used for training and 250 for testing. The entire dataset is divided into a training set of 75,750 images and a test set of 25,250 images.

[0088] Evaluation indicators:

[0089] The method proposed in this embodiment is compared with several state-of-the-art methods. Compared with many of the most advanced methods, the method in this embodiment achieves better results on both artificially synthesized noise datasets and real noise datasets. Some experimental results are shown in Tables 1 and 2.

[0090] Table 1 shows the test results on the CIFAR-10 dataset, which includes four types of artificially synthesized noise.

[0091]

[0092] Table 2 shows the test results on the CIFAR-10N dataset containing real noise.

[0093]

[0094] In this embodiment, a continuous learning approach is employed to address the challenges of learning with noisy labels. This learning is viewed as a continuous learning process encompassing multiple learning tasks. Each epoch corresponds to a task within the continuous learning process. By introducing the concept of parameter decomposition in continuous learning, the network parameters are decomposed into clean parameters and noisy parameters. Combining this with the DNN's tendency to prioritize easier tasks, a robust method based on time-varying regularization is designed. In the early stages of network training, the updating of clean parameters is strengthened while the updating of noisy parameters is restricted. As the number of network epochs increases, the constraints on clean parameter updates are gradually increased while the constraints on noisy parameter updates are decreased. This allows for dynamic and effective control of network parameter updates. The time-varying coefficients of CLTR regularization, derived from network predictions, dynamically reflect the interference of noisy labels on the training process and add regularization constraints in real time to control the direction of network parameter updates. The dynamic time-varying coefficients avoid the need for setting hyperparameters, reducing training difficulty and improving the network's applicability in practice.

[0095] Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-mentioned technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-mentioned technical features or their equivalent features without departing from the inventive concept. For example, technical solutions formed by substituting the above-mentioned features with technical features disclosed in this application (but not limited to) that have similar functions.

Claims

1. A robust classification method for noisy labeled images based on continuous learning time-varying regularization, characterized in that: Considering noisy label learning as a continuous learning process that includes multiple learning tasks, the network parameters are decomposed into two parts: clean and noisy. Dynamic regularization technology is used to control the update process of clean parameters and noisy parameters respectively. The specific steps are as follows: Use cross entropy loss to perform conventional training on the dataset with noisy labels, and use the validation dataset to test the accuracy of the model in each epoch to obtain the validation curve A. The validation curve is normalized and denormalized to obtain the increasing curve A1 and the decreasing curve A2. Take the epoch as the independent variable, and find the increasing function α(t) and decreasing function β(t) corresponding to the curve. By introducing the parameter processing method in catastrophic forgetting, the training parameters of the network are decomposed into clean parameters Θ clean and the noise parameter Θ noise , and use α(t) and β(t) as dynamic coefficients to impose different dynamic regularization constraints on clean parameters and noise parameters respectively.

2. The robust classification method for noisy labeled images based on continuous learning time-varying regularization according to claim 1, characterized in that: The target expression is: in, represents the total training target loss function, l is the cross entropy loss function, is the clean parameter before the current epoch, and ‖·‖2 represents the Euclidean norm.

3. The robust classification method for noisy labeled images based on continuous learning time-varying regularization according to claim 1, characterized in that: For clean parameters, update them correctly in the early stages of training, while restricting them from being updated in the later stages; For the noise parameters, their updates are limited in the early stages and gradually increased in the later stages.

4. The robust classification method for noisy labeled images based on continuous learning time-varying regularization according to claim 1, characterized in that: The process of obtaining the increasing and decreasing functions is as follows: In a given noisy labeled dataset Using the cross entropy loss function To train the network, the cross entropy loss function is defined as follows: Where N is the number of samples, K is the number of categories, Represents sample x i The corresponding true label.

5. The robust classification method for noisy labeled images based on continuous learning time-varying regularization according to claim 4, characterized in that: In the validation set Test the model and calculate the prediction accuracy A of the validation set t : Among them, 1 is the indicator function, which takes 1 when the prediction is correct and 0 otherwise; Get the verification accuracy curve: Where, E is the total number of epochs; Normalize the verification accuracy curve A to obtain the normalized curve A1: Denormalize the verification accuracy curve to obtain curve A2: A1′=-A1 The corresponding dynamic coefficient functions α(t) and β(t) are obtained through A1 and A2. The independent variable of the function is the epoch, and the dependent variable is the verification accuracy corresponding to each epoch.

Images