A method, system and storage medium for modeling and correcting noise labels in deep classification networks

Abstract

The present invention provides a deep classification network noise label modeling and correction method, system and storage medium, the deep classification network noise label modeling and correction method comprises the following steps: step 1, obtain a loss function curve; step 2, cluster fitting the loss function curve to obtain the label noise membership function value under single model training, and then return to step 1; step 3, multi-model noise label modeling; obtain the membership function values ​​of M models, perform multi-model confidence threshold averaging on the membership function values ​​of the M models, and obtain the final optimal membership function value; step 4, noise label correction and classification network training; according to the multi-model label modeling results, construct multiple label sets, and assign different weights to different label samples in the constructed label sets, obtain the corresponding designed weighted loss function to train the classification network. The beneficial effect of the present invention is: improving the training effect and classification accuracy of the classification network under noisy labels.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system and storage medium for modeling and correcting noise labels in a deep classification network. Background Art

[0002] Deep classification networks have been widely used in object classification tasks across a variety of image formats, including natural visual images, SAR images, and medical images. They have found widespread application in a variety of scenarios, including target monitoring, target recognition, image understanding, and medical diagnostic analysis. Due to the large number of parameters and strong model fitting capabilities of deep classification networks, they are susceptible to fitting noisy labels (erroneous labels), which impairs the training effectiveness of deep classification networks. Consequently, traditional classification network training methods are significantly affected by noisy labels. Training classification networks with noisy labels and correcting for noisy labels have become research hotspots in practical classification network applications.

[0003] Although there have been some studies on noisy labels, they mainly involve regularization constraints, label smoothing, and robust loss design. Summary of the Invention

[0004] In order to solve the problem in the prior art that the training effect of supervised learning classification networks is seriously impaired under noisy label datasets, the present invention provides a deep classification network noise label modeling and correction method.

[0005] The present invention provides a method for modeling and correcting noise labels in a deep classification network, comprising the following steps:

[0006] Step 1: Obtain the loss function curve;

[0007] Step 2: Perform cluster fitting on the loss function curve of step 1 to obtain the label noise membership function value under single model training, and then return to step 1; after looping through steps 1 and 2 M times, execute step 3 again;

[0008] Step 3: Multi-model noise label modeling; obtain the membership function values ​​of M models, perform multi-model confidence threshold averaging on the obtained membership function values ​​of the M models, and obtain the final optimal membership function value;

[0009] Step 4: Noise label correction and classification network training; Based on the multi-model label modeling results of step 3, multiple label sets are constructed, and different label samples in the constructed label sets are assigned different weights to obtain the corresponding designed weighted loss function to train the classification network.

[0010] As a further improvement of the present invention, the specific process of step 1 is as follows:

[0011] Assume that the target dataset is S, which contains the image set X n and the corresponding label set Y containing noisy labels n , the target image dataset S is used to train the deep classification network model. The loss function is defined as the cross entropy loss. The hyperparameter settings during training are the same as those of the deep network training model under supervised learning. During the training process, the image set X in each iteration stage is recorded. n The loss value corresponding to each image sample i in the image is obtained to obtain the corresponding loss function curve vector l i After the training, the corresponding loss function curve set {l i}.

[0012] As a further improvement of the present invention, the specific process of step 2 is as follows:

[0013] The loss function curve obtained in step 1 is used as the feature vector for label modeling; assuming that T represents the number of training iterations in the loss function curve acquisition phase in step 1, then any loss curve l i is a T-dimensional vector, the label noise modeling stage is modeled as an unsupervised clustering process, and the label noise modeling loss function is defined as The calculation formula is as follows:

[0014]

[0015] Where N is the number of loss function curves, q is the fuzzy index, c(k) represents the cluster center of the k-th cluster. In the noise label modeling stage, k has two values, 1 and 2, and it is stipulated that k = 1 corresponds to the cluster center being the noise-free label, u i (k) is the membership function, which represents the arbitrary loss function curve l i The degree of belonging to the kth cluster.

[0016] The loss function value is minimized through an alternating iterative process; first, the cluster center c(k) is initialized, and then the clusters and membership are updated alternately and iteratively using the following formula:

[0017]

[0018]

[0019] When the optimization iterative process finally converges, the optimal cluster center c will be obtained. * (1) and c * (2) and the noise membership function of each loss curve label c * (1) corresponds to the noise-free label, c *(2) Corresponding to the noise label, N is the number of loss function curves, i is the subscript of the loss function sequence, and the membership function values ​​of all samples corresponding to the loss curve constitute the set of label noise membership function values ​​under single model training

[0020] As a further improvement of the present invention, the step three includes:

[0021] Step 1: Obtain a set of multi-model membership function values; use the same training set to train different network models each time to obtain different noise label modeling results, i.e., cluster center c * m (1) and c * m (2) and the set of membership function values ​​of each loss function

[0022] Step 2: Calculate the multi-model label noise average using a confidence threshold-based method. The specific calculation formula is as follows:

[0023]

[0024] in,

[0025]

[0026]

[0027] It represents the threshold-based average of the membership function corresponding to the loss curve i under the optimal clustering result after the fitting convergence in step 2 under multiple models, It represents the arithmetic average of the membership function corresponding to the loss curve i under the optimal clustering result after the fitting convergence in step 2 under multiple models, is the indicator function, when When the function value is 1, otherwise it is 0, j represents the subscript index of all loss curves, p i Represents the variable p corresponding to the i-th training loss function curve;

[0028] The noise label modeling results under multiple models are calculated as:

[0029]

[0030] Indicates that the label corresponding to sample i is modeled as a noise-free label, Indicates that the label corresponding to sample i is modeled as a noise label.

[0031] As a further improvement of the present invention, in step 4, according to the noise label modeling result and multi-model membership function values Construct a high-confidence error-free label set, a high-confidence error label set, and a low-confidence label modeling set. During the correction training process, different weights are assigned to different label samples in the constructed high-confidence error-free label set, high-confidence error label set, and low-confidence label modeling set. Combined with the mixup label construction method, a weighted loss function is designed to train the classification network.

[0032] The present invention also discloses a deep classification network noise label modeling and correction system, comprising: a memory, a processor, and a computer program stored on the memory, wherein the computer program is configured to implement the steps of the deep classification network noise label modeling and correction method of the present invention when called by the processor.

[0033] The present invention also discloses a computer-readable storage medium, which stores a computer program. The computer program is configured to implement the steps of the deep classification network noise label modeling and correction method of the present invention when called by a processor.

[0034] The beneficial effects of the present invention are as follows: a deep classification network noise label modeling and correction method disclosed by the present invention fully exploits the characteristics contained in the loss function curve under multiple models, and models noise labels by analyzing the label noise confidence under multiple models, thereby realizing the correction of noise labels and improving the training effect and classification accuracy of the classification network under noisy labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of the deep classification network noise label modeling and correction method of the present invention. DETAILED DESCRIPTION

[0036] The present invention discloses a method for modeling and correcting noise labels in a deep classification network. The method is a method for modeling and correcting noise labels in a deep classification network based on multi-model loss function curve fitting. The method is a new noise label correction method suitable for training deep classification networks in situations where a certain proportion of noise labels exist. The method can effectively train a deep classification network in situations where noise labels are included in the training data, thereby obtaining a reliable classification network model. At the same time, the method can model and correct the noise labels, reduce the impact of the noise labels on the network model, and improve the classification accuracy of the deep classification network in the presence of noise labels, thereby meeting the network training requirements for classification tasks involving a large number of noise labels in SAR images and natural visual images.

[0037] like Figure 1 As shown, the present invention discloses a method for modeling and correcting noise labels in a deep classification network, comprising the following steps:

[0038] Step 1: Loss function curve acquisition stage; obtain the loss function curve;

[0039] Assume that the target dataset is S, which contains the image set X n and the corresponding label set Y containing noisy labels n The target image dataset S is used to train the deep classification network model, and the loss function is defined as the cross entropy loss. The hyperparameter settings during training are the same as those of the deep network training model under supervised learning. During the training process, the image set X in each iteration stage is recorded. n The loss value corresponding to each image sample i in the image is obtained to obtain the corresponding loss function curve vector l i ; After the training is completed, the corresponding loss function curve set {l i}.

[0040] Step 2: Single model label noise modeling stage based on loss function curve; cluster fitting is performed on the loss function curve of step 1 to obtain the label noise membership function value under single model training, and then return to step 1.

[0041] In this stage, the loss function curve obtained in step 1 is used as a feature vector for label modeling. The specific steps are as follows: Assume that T represents the number of training iterations in the loss function curve acquisition stage in step 1, then any loss curve l i For a T-dimensional vector, the present invention models the label noise modeling stage as an unsupervised clustering process and defines the label noise modeling loss function as The specific calculation is shown in formula (1):

[0042]

[0043] Where N is the number of loss function curves and q is the fuzzy index. c(k) represents the cluster center of the k-th cluster. In the noise label modeling stage, k has two values, 1 and 2, and it is stipulated that k = 1 corresponds to the cluster center being the noise-free label. u i (k) is the membership function, which represents the arbitrary loss function curve l i The degree of belonging to the kth cluster.

[0044] Minimizing the loss function value can be achieved through an alternating iterative process. First, the cluster value c(k) is initialized, and then the clusters and membership are updated alternately and iteratively using the following formula:

[0045]

[0046]

[0047] When the optimization iterative process finally converges, the optimal cluster center c will be obtained.* (1) (corresponding to the noise-free label) and c * (2) (corresponding to the noise label) and the noise membership function of each loss curve label The membership function values ​​of the loss curves corresponding to all samples constitute the set of label noise membership function values ​​under single model training q is the fuzzy index.

[0048] After looping through steps 1 and 2 M times, execute step 3 again, where M is a specified number, for example, M is 3.

[0049] Step 3: Multi-model noise label modeling stage; obtain the membership function values ​​of M models, perform multi-model confidence threshold averaging on the obtained membership function values ​​of the M models, and obtain the final optimal membership function value.

[0050] Step three includes:

[0051] Step 1: Obtain a set of multi-model membership function values.

[0052] Construct multiple network structure models and record the model numbers as 1, 2, ..., m, ..., M. For multiple models, loop through steps 1 and 2, and use the same training set to train different network models each time to obtain different noise label modeling results, i.e., cluster center c * m (1) and c * m (2) and the set of membership function values ​​of each loss function There are many ways to build multiple models: (1) select different depth versions of the same network model, such as Resnet18, Resnet50, Resnet101, etc.; (2) use network pruning methods such as dropout to randomly modify the network to build different models.

[0053] Step 2: After obtaining the multi-model membership function value set, the multi-model label noise average is calculated using a confidence threshold-based method. The specific calculation is shown in formula (4):

[0054]

[0055] in,

[0056]

[0057] Represents the arithmetic mean of the membership functions of M models, where M is the number of models in the multi-model system. represents the membership function of curve i after the cluster fitting convergence obtained in step 2 under any model m;

[0058] It represents the threshold-based average of the membership function corresponding to the loss curve i under the optimal clustering result after the fitting convergence in step 2 under multiple models, It represents the arithmetic average of the membership function corresponding to the loss curve i under the optimal clustering result after the fitting convergence in step 2 under multiple models, is the indicator function, when The function value is 1 when , otherwise it is 0. represents the product of the indicator function and the arithmetic mean (Formula 5); p i and T are two parameters of the Sharpen function, T is the hyperparameter in the sharpen function, j represents the subscript index of all loss curves, p i 1 / T Indicates p i T / 1, p j 1 / T Indicates p j The first power of T, i represents the i-th training loss function curve, p i represents the variable p corresponding to the i-th training loss function curve, specifically the membership function u in Formula 4;

[0059] The noise label modeling results under multiple models can be calculated as:

[0060]

[0061] Indicates that the label corresponding to sample i is modeled as a noise-free label, Indicates that the label corresponding to sample i is modeled as a noise label; k takes the value of 1 or 2, 1 corresponds to a noiseless label, and 2 corresponds to a noise label; the difference between k* and k is that k* represents the convergence value after the model is optimized.

[0062] Step 4: Noise label correction and classification network training stage.

[0063] Based on the multi-model label modeling results, the noise label modeling results can be and multi-model membership function values This constructs a set of high-confidence error-free labels, a set of high-confidence error labels, and a set of low-confidence label modeling. During the corrective training process, different weights are assigned to samples with different labels in the three sets. Combined with the mixup label construction method, a weighted loss function is designed to train the classification network. Mixup is a data augmentation method.

[0064] As an embodiment of the present invention, taking the MSTAR dataset as an example, 80% of the labels in the original dataset training samples are assigned random values, thereby obtaining a training dataset S with a noise label ratio of about 80%.

[0065] The residual network is selected as the deep classification network result, and the multiple models are set to three residual networks of different depths: Resnet18, Resnet50, and Resnet101.

[0066] Step 1 and step 2 are executed three times in a loop. The implementation method is as follows:

[0067] Using cross entropy loss as training loss, the training parameters are set as follows: batch size is 16, learning rate is 0.0001, and the network is trained separately for 300 generations. The loss function curve sets under different models are obtained respectively. Obtain the set of label noise clustering membership function values ​​through loss function curve modeling

[0068] Execute step 3 to fuse the multi-model membership values ​​and obtain the fused sample label noise membership value

[0069] Execute step 4 and use the membership threshold (the threshold can be set to 0.8) according to The samples are divided into three groups according to the value size, including The corresponding high confidence error-free label set, The corresponding high confidence error label set and The corresponding low-confidence label set. The label training weight in the high-confidence error-free label set is set to 1, the label training weight in the high-confidence error label set is set to 0, and the label training weight in the low-confidence label set is set to 0.5. The network is trained using the mixup sample mixing technique and the cross-entropy loss to achieve label correction training for the network.

[0070] The present invention discloses a deep classification network noise label modeling and correction system, wherein the computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the deep classification network noise label modeling and correction method of the present invention when called by a processor.

[0071] The present invention discloses a computer-readable storage medium, which stores a computer program. The computer program is configured to implement the steps of the deep classification network noise label modeling and correction method of the present invention when called by a processor.

[0072] The present invention does not require prior knowledge of the ratio of noise labels and is still applicable when the noise ratio is large. It is also generally applicable to any classification network and any image target classification task.

[0073] Beneficial effects of the present invention: The method for modeling and correcting noise labels in a deep classification network disclosed in the present invention fully exploits the characteristics contained in the loss function curve under multiple models, and models noise labels by analyzing the label noise confidence under multiple models, thereby achieving correction of noise labels and improving the training effect and classification accuracy of the classification network under noisy labels.

[0074] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for modeling and correcting noise labels in deep classification networks, characterized by: The following steps are involved: Step 1: Obtain the loss function curve; Step 2: Perform cluster fitting on the loss function curve of step 1 to obtain the label noise membership function value under single model training, and then return to step 1; after looping through steps 1 and 2 M times, execute step 3 again; Step 3: Multi-model noise label modeling; obtain the membership function values ​​of M models, perform multi-model confidence threshold averaging on the obtained membership function values ​​of the M models, and obtain the final optimal membership function value; Step 4: Noise label correction and classification network training. Based on the multi-model label modeling results of step 3, multiple label sets are constructed, and different label samples in the constructed label sets are assigned different weights. The corresponding designed weighted loss function is obtained to train the classification network. The step three includes: Step 1: Obtain a set of multi-model membership function values; use the same training set to train different network models each time to obtain different noise label modeling results, i.e., cluster center c * m (1) and c * m (2) and the set of membership function values ​​of each loss function Step 2: Calculate the multi-model label noise average using a confidence threshold-based method. The specific calculation formula is as follows: in, It represents the threshold-based average of the membership function corresponding to the loss curve i under the optimal clustering result after the fitting convergence in step 2 under multiple models, It represents the arithmetic average of the membership function corresponding to the loss curve i under the optimal clustering result after the fitting convergence in step 2 under multiple models, is the indicator function, when The function value is 1 when , otherwise it is 0; j represents the subscript index of all loss curves, p i Represents the variable p corresponding to the i-th training loss function curve; The noise label modeling results under multiple models are calculated as: Indicates that the label corresponding to sample i is modeled as a noise-free label, Indicates that the label corresponding to sample i is modeled as a noise label.

2. The method for modeling and correcting noise labels in a deep classification network according to claim 1, characterized in that: The specific process of step 1 is as follows: Assume that the target dataset is S, which contains the image set X n and the corresponding label set Y containing noisy labels n , the target image dataset S is used to train the deep classification network model. The loss function is defined as the cross entropy loss. The hyperparameter settings during training are the same as those of the deep network training model under supervised learning. During the training process, the image set X in each iteration stage is recorded. n The loss value corresponding to each image sample i in the image is obtained to obtain the corresponding loss function curve vector l i After the training, the corresponding loss function curve set {l i }.

3. The method for modeling and correcting noise labels in a deep classification network according to claim 1, characterized in that: The specific process of step 2 is as follows: The loss function curve obtained in step 1 is used as the feature vector for label modeling; assuming that T represents the number of training iterations in the loss function curve acquisition phase in step 1, then any loss curve l i is a T-dimensional vector, the label noise modeling stage is modeled as an unsupervised clustering process, and the label noise modeling loss function is defined as The calculation formula is as follows: Where N is the number of loss function curves, q is the fuzzy index, c(k) represents the cluster center of the k-th cluster. In the noise label modeling stage, k has two values, 1 and 2, and it is stipulated that k = 1 corresponds to the cluster center being the noise-free label, u i (k) is the membership function, which represents the arbitrary loss function curve l i The degree of belonging to the kth cluster; The loss function value is minimized through an alternating iterative process; first, the cluster center c(k) is initialized, and then the clusters and membership are updated alternately and iteratively using the following formula: When the optimization iterative process finally converges, the optimal cluster center c will be obtained. * (1) and c * (2) and the noise membership function of each loss curve label c * (1) corresponds to the noise-free label, c * (2) Corresponding to the noise label, N is the number of loss function curves, i is the subscript of the loss function sequence, and the membership function values ​​of all samples corresponding to the loss curve constitute the set of label noise membership function values ​​under single model training 4. The method for modeling and correcting noise labels in a deep classification network according to claim 1, characterized in that: In the fourth step, the noise label modeling results are and multi-model membership function values Construct a high-confidence error-free label set, a high-confidence error label set, and a low-confidence label modeling set. During the correction training process, different weights are assigned to different label samples in the constructed high-confidence error-free label set, high-confidence error label set, and low-confidence label modeling set. Combined with the mixup label construction method, a weighted loss function is designed to train the classification network.

5. A deep classification network noise label modeling and correction system, characterized by: include: A memory, a processor, and a computer program stored on the memory, wherein the computer program is configured to implement the steps of the deep classification network noise label modeling and correction method according to any one of claims 1 to 4 when called by the processor.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the deep classification network noise label modeling and correction method described in any one of claims 1 to 4 when called by a processor.

Images