A two-step learning method for noisy labeled samples based on contrastive loss

By combining contrast loss and semi-supervised learning in the two-step noisy label sample learning method, and optimizing sample selection and robustness training in step by step, the problem of poor performance of multi-category noise data sets in the existing technology is solved, significantly improving the model robustness and classification performance.

CN117952184BActive Publication Date: 2024-11-22JIANGSU OPEN UNIVERSITY (THE CITY VOCATIONAL COLLEGE OF JIANGSU)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410151324.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-02
Publication Date
2024-11-22
Estimated Expiration
2044-02-02

AI Technical Summary

Technical Problem

The prior art performs poorly when dealing with multi-category noise data sets, and a one-step framework method couples sample selection and robust training, increasing optimization difficulty and model training complexity.

Method used

Using a two-step noisy label sample learning method based on contrast loss, the model is pre-trained by a robust loss function to reduce the sensitivity of noise category, and in two steps, the comparison loss and semi-supervised learning techniques are combined to optimize sample selection in steps. and robustness training.

Benefits of technology

It improves the robustness and classification performance of the model, and is suitable for multi-category noise datasets, including category-related and feature-related noise, significantly improving the effectiveness of comparison learning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117952184B_ABST
    Figure CN117952184B_ABST
Patent Text Reader

Abstract

The present invention proposes a two-step noisy labeled sample learning method based on contrastive loss. This method aims to solve the problems of the existing two-step noisy sample learning method LongReMix, such as poor performance of noisy data sets with a large number of categories and sensitivity to noise types. Firstly, a robustness loss function is introduced to pre-train two models to reduce the sensitivity to noise categories. Secondly, contrastive loss is introduced in both steps in combination with SSL technology, so that the model can learn more feature information of unlabeled samples, thereby improving the robustness of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a two-step noisy label sample learning method based on contrast loss. Background Art

[0002] One of the key factors for the great success of Deep Neural Networks (DNN) in the field of computer vision can be attributed to large-scale and accurately labeled datasets. However, the collection of such datasets is very expensive and inevitably introduces unclean labels (noisy labels). Existing training methods can easily cause DNN to fall into overfitting, resulting in performance degradation, so learning with noisy labels (LNL) has become a huge challenge.

[0003] Han et al. (B. Han, Q. M. Yao, X. R. Yu, G. Niu, M. Xu, W. H., I. W. Tsang, M. Sugiyama, "Co-teaching: Robust training of deep neural networks with extremely noisy labels," in Proc. IEEE Conf. Adv. Neural Inf. Process. Sys., Vol. 31, 2018.) found from experience that samples with small losses during training are more likely to contain clean labels. Therefore, two models of the same structure are trained at the same time to filter samples with clean labels according to the sample loss, but the test performance of the trained models is low. Wei et al. (HX Wei, L. Feng, XY Chen, B. An, "Combating noisy labels by agreement: A joint training method with co-regularization," in Proc. IEEE Conf. on Comput. Vis. and Pattern Recognit., 2020.) introduced a joint regularization term to train two models, so that the two models tend to be consistent for the input of the same sample, thereby improving the model test performance, but it performs poorly on real noise data sets. Since the above method only selects clean samples for training and directly discards the remaining samples, the model cannot learn enough feature information, resulting in a decrease in performance. Therefore, Li et al. (J. Li, R. Socher, S. Hoi, "Divide Mix: Learning with noisy labels as semi-supervised learning," in Proc. Int. Conf. on Learn. Represent., 2020.) removed the labels of the remaining samples after screening, that is, the unlabeled set, and then introduced the existing semi-supervised learning method (SSL) to train the two filtered subsets (clean sample set and unlabeled set), and achieved good results. However, this method performs poorly under high noise ratios and is not very suitable for data sets with a large number of categories.Li et al. (SKLi, XBXia, SMGe, TLLiu, "Selective-supervised contrastive learning with noisy labels," in Proc.IEEE Conf.onComput.Vis.and Pattern Recognit., 2022, pp.316-325, doi: 10.1109 / CVPR52688.2022.00041.) introduced contrastive learning into the field of noisy label learning and combined it with SSL technology to achieve certain results, but it requires adjusting more parameters, so it is not very applicable in practice. Sheng Mengmeng et al. ("Image recognition method and system with noisy labels based on category-balanced sample selection CN202311453720.9") first selected a subset of clean samples in the dark based on the loss shaixu, and then enhanced the confidence of the samples in the subset to obtain an enhanced subset for training, which made certain progress, but this method performed poorly under high noise ratios. Cheng Lechao et al. (“A method and device for image classification based on class prototype cleaning and denoising CN202311397429.4”) used pseudo-labels to calculate category prototypes, and used the obtained category prototypes to divide the data set, and then used the divided data subsets to apply different losses for training to improve performance. However, considering that the pseudo-labels generated by the model under high noise ratios are usually inaccurate, this method has shortcomings in practical applications. In addition, Zhang Qian et al. (“A noisy label learning method based on category balance and cross-merging strategy CN202311277266.6”) proposed a cross-merging training strategy, which periodically merges the two model parameters and redistributes them to the two models during training, which can significantly improve the model accuracy. It has made good progress in combination with the category balance strategy, but this method takes a long time to train. The above-mentioned noise learning methods trained using SSL technology all solve the two optimization objectives of sample selection and robustness training in the same iteration round (epoch), so they are called one-step frameworks. The one-step framework method needs to solve two optimization objectives in the same epoch. Since the two objectives are coupled with each other, it complicates the model training process and increases the difficulty of optimization.Therefore, Cordeiro et al. (FR Cordeiro, R. Sachdeva, V. Belgiannis, I. Reid, G. Carneiro, "LongReMix: Robust learning with high confidence samples in a noisy label environment," in Pattern Recognit., vol. 133, 2023, doi: 10.1016 / j.patcog.2022.109013) decoupled sample selection and robustness training into two separate optimization objectives, and optimized them in two steps. This method was inspired by the DivideMix method of Li et al. In the first step, the most refined screening goal of clean samples is solved, and in the second step, the set of clean samples obtained in the first step is expanded to improve the model robustness and test performance. However, since this method is directly improved on the basis of DivideMix, it inherits its poor performance on high noise data sets and data sets with a large number of categories. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a two-step noisy labeled sample learning method based on contrastive loss. The method addresses the problems existing in the existing two-step noisy sample learning method LongReMix, such as poor performance of noise data sets with a large number of categories and sensitivity to noise types. Firstly, a robustness loss function is introduced to pre-train two models to reduce the sensitivity to noise categories. Secondly, contrastive loss is introduced in both steps in combination with SSL technology, so that the model can learn more feature information of unlabeled samples to improve the robustness of the model.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A two-step noisy labeled sample learning method based on contrastive loss, including:

[0007] Step 1: Select two DNN models with the same structure and define them as a first DNN model and a second DNN model; perform a first pre-training and a first formal training on the two DNN models based on a given initial noise label data set; the initial noise label data set consists of image samples and noisy labels of the image samples;

[0008] After obtaining the first DNN model and the second DNN model with preliminary discrimination ability through the first pre-training, in the first formal training, the first DNN model and the second DNN model are used to screen clean samples from the initial noise label data set to construct a clean sample set, and the index of the clean sample is marked as a True value;

[0009] Step 2: reinitialize the two DNN models, and perform a second pre-training and a second formal training on the two initialized DNN models based on a given initial noise label data set;

[0010] After obtaining the first DNN model and the second DNN model with preliminary discrimination ability through the second pre-training, in each round of iteration of the second formal training:

[0011] Using the first DNN model to label the samples of the initial noisy label data set, putting all samples indexed as True values ​​or belonging to the clean sample set in the labeling results of the first DNN model into the first labeled set, and putting the remaining samples of the initial noisy label data set into the first unlabeled set;

[0012] Re-label the samples of the initial noisy label data set using the second DNN model, put the samples whose index is marked as True or belongs to the clean sample set in the labeling result of the second DNN model into the second labeled set, and put the remaining samples of the initial noisy label data set into the second unlabeled set;

[0013] For the first DNN model, after semi-supervised and contrastive loss calculations are performed based on the second labeled set and the second unlabeled set, back-propagation is performed to update the parameters of the first DNN model;

[0014] For the second DNN model, after semi-supervised and contrastive loss calculations are performed based on the first labeled set and the first unlabeled set, back propagation is performed to update the parameters of the second DNN model;

[0015] After the training is completed, two final DNN models are obtained.

[0016] To optimize the above technical solutions, the specific measures taken also include:

[0017] Furthermore, in step 1.1, the maximum total number of iterations is preset. And initialize the count of the current total iteration round t;

[0018] The maximum total number of iterations The target iteration number for the first pre-training and the sum of the target iteration rounds of the first formal training;

[0019] Step 1.2: Use the GCE loss function based on the given initial noise label dataset Performing a first pre-training on the two DNN models, so that the first DNN model and the second DNN model have preliminary discrimination capabilities through the first pre-training;

[0020] Among them, x i represents the i-th image sample, Represents image sample x i The noisy labels, n is the number of samples in the initial noisy label dataset;

[0021] At the end of each iteration of the first pre-training, the initial noise label dataset is calculated The cross entropy loss of the samples in the two DNN models is used to estimate the sample loss distribution using GMM. The samples whose distribution probability is greater than or equal to the preset threshold will have their indexes marked as True values. The indexes of the remaining samples are marked as False values, and the marking results are saved in the first historical sequence S1 and the second historical sequence S2 respectively according to the order of iteration rounds;

[0022] Step 1.3: After the first pre-training is completed, the two DNN models with preliminary discrimination capabilities are formally trained for the first time. Each iteration of the first formal training includes the following steps:

[0023] Step 1.31: For the first DNN model, first calculate the initial noise label dataset The cross entropy loss of each sample in the two DNN models is used to estimate the current loss distribution probability w of the sample using GMM. i ,Will Satisfy w i The indices of samples with ≥τ are marked as True values. The indexes of the remaining samples are marked as False values, τ is the preset threshold value, and then the marking results are saved in the first historical sequence S1 in the order of iteration rounds; based on the marking results in the current first historical sequence S1, the initial noise label data set is Select samples that are continuously marked as True and put them into the first labeled set Will The remaining samples are placed in the first unlabeled set

[0024] Step 1.32: For the second DNN model, first calculate the cross entropy loss of each sample in the initial noise label data set on the two DNN models, and use GMM to estimate the current loss distribution probability w of the sample i ,Will Satisfy wi The sample indices with ≥τ are marked as True values, The indexes of the remaining samples are marked as False, and τ is the preset threshold value; the marking results are saved in the second historical sequence S2 in the order of iteration rounds, and based on the marking results in the current second historical sequence S2, the initial noise label data set is Select samples that are continuously marked as True and put them into the second labeled set and will The remaining samples are placed in the second unlabeled set

[0025] Step 1.33: For the first DNN model, based on the second labeled set and the second unlabeled set After calculating the semi-supervised loss, the second unlabeled set Compute the contrast loss calculation based on the samples in the above code, and propagate the overall loss back to update the first DNN model parameters;

[0026] Step 1.34: The second DNN model is based on the first labeled set and the first unlabeled set After calculating the semi-supervised loss, the first unlabeled set Calculate the contrast loss and backpropagate the overall loss to update the second DNN model parameters;

[0027] Step 1.35: Add 1 to the t value to determine whether the count of the current total iteration round t is less than or equal to If satisfied, proceed to the next iteration of the first formal training. If not satisfied, proceed to step 1.4.

[0028] Step 1.4: Select the last consecutive In the iterations of rounds, all samples marked as True values ​​are put into the clean sample set D c , the formula is as follows:

[0029]

[0030] Wherein, state[j][i] represents the marking state of the i-th sample of the j-th sequence of the first historical sequence S1.

[0031] Furthermore, step 2 specifically includes:

[0032] Step 2.1: Preset the maximum total number of iterations And initialize the count of the current total iteration round t;

[0033] The maximum total number of iterations The target iteration round for the second pre-training and the sum of the target iteration rounds of the second formal training;

[0034] Step 2.2: Reinitialize the two DNN models using the GCE loss function based on the initial noisy label dataset The two DNN models are pre-trained for the second time, and the first DNN model and the second DNN model have preliminary discrimination capabilities through the second pre-training; wherein, x i represents the i-th image sample, Represents image sample x i The noisy labels, n is the number of samples in the initial noisy label dataset;

[0035] Step 2.3: After the second pre-training is completed, the two DNN models with preliminary discrimination capabilities are formally trained for the second time. Each iteration of the second formal training includes the following steps:

[0036] Step 2.31: For the first DNN model, calculate the initial noise label dataset The cross entropy loss of each sample in the two DNN models is used, and GMM is used to estimate the current loss distribution probability w of the sample. i ,Will Satisfy w i The sample index ≥τ is marked as True value, τ is the preset threshold value;

[0037] Initialize the first labeled set and the first unlabeled set After that, only the samples marked with True values ​​are put into the first labeled set The samples marked as True here include the clean samples screened in step 1, and then The indices of the remaining samples are marked as False values ​​and put into the first unlabeled set The specific formula is as follows:

[0038]

[0039] Step 2.32: For the second DNN model, calculate the cross entropy loss of each sample in the initial noise label data set on the two DNN models, and use GMM to estimate the current loss distribution probability w of the sample i ,Will Satisfy w i ≥τ sample indexes are marked as True values;

[0040] Initialize the second labeled set and the second unlabeled set After that, only the samples marked as True are put into the second labeled set The samples marked as True here include the clean samples screened in step 1, and then The indices of the remaining samples are marked as False values ​​and put into the second unlabeled set The specific formula is as follows:

[0041]

[0042] Step 2.33: For the first DNN model, based on the second labeled set and the second unlabeled set After calculating the semi-supervised loss, the second unlabeled set Calculate the contrast loss for the samples in and backpropagate the overall loss to update the first DNN model parameters;

[0043] The calculation formula of the overall loss is as follows:

[0044] L total =L ssl +λ cl ·L cl

[0045] Where, L total is the overall loss, λ cl is the weight coefficient of contrast loss, L ssl is the semi-supervised loss, L cl is the contrast loss;

[0046] Step 2.34: For the second DNN model, based on the first labeled set and the first unlabeled set After calculating the semi-supervised loss, the first unlabeled set After calculating the contrast loss of the samples in, the overall loss is back-propagated to update the parameters of the second DNN model;

[0047] Step 2.35: Add 1 to the t value to determine whether the current total iteration round t is less than or equal to If it is satisfied, the next round of iteration of the second formal training will be continued. If it is not satisfied, the training will be terminated to obtain two final DNN models.

[0048] Furthermore, in step 1.2, the GCE loss function is specifically:

[0049]

[0050] In the formula, l GCE is the GCE loss function, p(x i ) represents the classifier for the i-th image sample xi The predicted value of is the value after the softmax operation;

[0051] The calculation formula of the cross entropy loss is as follows:

[0052]

[0053] In the formula, l ce represents the cross entropy loss.

[0054] Furthermore, in step 1.31, the samples marked as True in the last K consecutive iterations in the first historical sequence S1 are selected and put into the first labeled set according to the following formula: Put all other samples into the first unlabeled set

[0055]

[0056] Wherein, state[j][i] represents the marking state of the i-th sample of the j-th sequence of the first historical sequence S1, and K is the number of times the preset sample is continuously marked as True.

[0057] Furthermore, in step 1.32, only the samples marked as True in the last K consecutive iterations in the second historical sequence S2 are selected and put into the second labeled set according to the following formula: Put all other samples into the second unlabeled set

[0058]

[0059] Wherein, state[j][i] represents the marking state of the i-th sample of the j-th sequence of the second historical sequence S2, and K is the number of times the preset sample is continuously marked as True.

[0060] Furthermore, step 1.33 is specifically as follows:

[0061] For the first DNN model, the SSL loss function is used based on the second labeled set and the second unlabeled set The semi-supervised loss L is calculated according to the following formula ssl ;

[0062]

[0063] Among them, l x is the classification loss function, l u is the absolute mean loss, l reg is the regularization term, λ u and λ regis the preset weight coefficient;

[0064] For the second unlabeled set For each sample in , the contrast loss is calculated according to the following formula:

[0065]

[0066]

[0067] Among them, L cl represents the contrast loss, l(i, j) represents the similarity value between the i-th sample and the j-th sample, z i Represents sample x i The normalized eigenvalue of j Represents sample x j The normalized eigenvalue of k Represents sample x k The normalized eigenvalue of , and k≠j, and δ is the preset scaling factor; and The second unlabeled set The unlabeled set after each sample in the strong data enhancement transformation s_da(·) is twice is expressed as follows:

[0068]

[0069] In the formula, b represents the size of minibatch;

[0070] Finally, the overall loss L for backpropagation is total Calculated according to the following formula:

[0071] L total =L ssl +λ cl ·L cl

[0072] where λ cl is the weight coefficient.

[0073] Furthermore, GMM is used to estimate the current loss distribution probability w of the sample i The method is:

[0074] Use a two-component GMM to estimate the loss distribution of the sample, select the component with the smaller mean as the estimated probability distribution model of the sample containing the clean label, and determine the loss distribution probability w of each sample through the probability distribution model i .

[0075] The beneficial effects of the present invention are:

[0076] The method proposed in the present invention decouples the two objectives of sample selection and robustness training and optimizes them in steps. Combined with the introduced contrast loss, better classification performance can be obtained than similar methods. It is also applicable to various types of noise, including category-related noise, feature-related noise and real noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 Step 1 Specific learning process.

[0078] Figure 2 Step 2 Specific learning process.

[0079] Figure 3 The overall process of the two-step noisy label sample learning method based on contrastive loss. DETAILED DESCRIPTION

[0080] The present invention will now be described in further detail with reference to the accompanying drawings.

[0081] The present invention proposes a two-step noisy label sample learning method based on contrast loss. The overall process of the method is as follows: Figure 3 As shown, including:

[0082] Step 1: Select two DNN models with the same structure and define them as a first DNN model and a second DNN model; perform a first pre-training and a first formal training on the two DNN models based on a given initial noise label data set; the initial noise label data set consists of image samples and noisy labels of the image samples;

[0083] After obtaining the first DNN model and the second DNN model with preliminary discrimination ability through the first pre-training, in the first formal training, the first DNN model and the second DNN model are used to screen clean samples from the initial noise label data set to construct a clean sample set, and the index of the clean sample is marked as a True value;

[0084] The process of step 1 is as follows Figure 1 As shown, specifically:

[0085] Step 1.1: Preset the maximum total number of iterations And initialize the count of the current total iteration round t;

[0086] The maximum total number of iterations The target iteration number for the first pre-training and the sum of the target iteration rounds of the first formal training;

[0087] Step 1.2: Use the GCE loss function, i.e., formula (1), based on the initial noisy label dataset The first pre-training of two DNN models is performed, where xi represents the i-th image sample, Represents image sample x i The noisy labels are represented in a one-hot matrix format, with a certain probability that they are not equal to the actual category of the sample; n is the number of samples in the initial noisy label dataset;

[0088]

[0089] In the formula, l GCE is the GCE loss function, p(x i ) represents the classifier for the i-th image sample x i The predicted value of is the value after the softmax operation;

[0090] The first pre-training enables the first DNN model and the second DNN model to have preliminary discrimination capabilities.

[0091] At the end of each iteration of the first pre-training, the initial noise label dataset is calculated according to formula (2): The cross entropy loss of samples in two DNN models:

[0092]

[0093] In the formula, l ce represents the cross entropy loss.

[0094] A two-component GMM model is used to estimate the loss distribution of the sample, and the component with the smaller mean is selected as the estimated probability distribution model of the sample containing the clean label. For the distribution probability w i The sample indexes greater than or equal to the preset threshold τ are marked as True values, and the rest are marked as False values, and the marking results are saved in the first historical sequence S1 and the second historical sequence s2 respectively according to the order of the pre-training iteration rounds;

[0095] Step 1.3: After the first pre-training is completed, the two DNN models with preliminary discrimination capabilities are formally trained for the first time. Each iteration of the first formal training includes the following steps:

[0096] Step 1.31: For the first DNN model, first calculate the cross entropy loss of each sample in the initial noise label data set on the two DNN models, and use GMM to estimate the sample loss distribution probability w i , will satisfy w i The sample index with ≥τ is marked as True value, The indexes of the remaining samples are marked as False values, and τ is a preset threshold value; and then the marking results are saved in the first historical sequence S1 according to the order of iteration rounds.

[0097] Based on the labeling results in the first historical sequence S1, the initial noise label dataset Divide into the first labeled set and the first unlabeled set The specific process is:

[0098] According to the following formula, only those samples marked as True in the last K consecutive iterations in the first historical sequence S1 are selected and put into the first labeled set The remaining samples are placed in the first unlabeled set

[0099]

[0100] Wherein, state[j][i] represents the marking state of the i-th sample of the j-th sequence of the first historical sequence S1. If the mark is True, the current value state[j][i]=1, otherwise it is 0.

[0101] Step 1.32: For the second DNN model, first calculate the cross entropy loss of each sample in the initial noise label data set on the two DNN models, and use GMM (Gaussian Mixture Model) to estimate the sample loss distribution probability w i ,Will Satisfy w i The sample indices with ≥τ are marked as True values, The rest of the samples are marked as False, τ is the preset threshold value; the marking results are saved in the second historical sequence S2 in the order of iteration rounds, and based on the marking results in the current second historical sequence S2, the initial noise label data set is Select samples that are continuously marked as True and put them into the second labeled set and will The remaining samples are placed in the second unlabeled set The formula is as follows:

[0102]

[0103] Wherein, state[j][i] represents the marking state of the i-th sample of the j-th sequence of the second historical sequence S2.

[0104] Step 1.33: For the first DNN model, based on the second labeled set and the second unlabeled set After calculating the semi-supervised loss, the second unlabeled set The sample calculation in the contrast loss calculation is used to propagate the overall loss back to update the first DNN model parameters; specifically:

[0105] For the first DNN model, the SSL loss function is used based on the second labeled set and the second unlabeled set According to formula (5), the semi-supervised loss L is calculated ssl ;

[0106]

[0107] Among them, l x is the classification loss function, l u is the absolute mean loss, l reg is the regularization term, λ u and λ reg is the preset weight coefficient; represents a labeled set. The first labeled set or the second labeled set is substituted into formula (5) as needed. represents the unlabeled set, and the first unlabeled set or the second unlabeled set is substituted into formula (5) as needed.

[0108] In addition, consistent with the SimCLR contrast loss method, for the second unlabeled set For each sample in the example, the present invention additionally introduces a mapping head g(.; ψ), where ψ is a mapping layer parameter, which can transform the high-dimensional feature h extracted by the feature extractor h(·; φ) into i =h(x i ; φ) is mapped to a low-dimensional space to reduce computing resources, φ is the parameter of the feature extractor, and the contrast loss is calculated according to the following formula to improve the feature extraction and clustering capabilities of the model:

[0109]

[0110]

[0111] Among them, L cl represents the contrast loss, l(i, j) represents the similarity value between the i-th sample and the j-th sample, z i Represents sample x i The normalized eigenvalue of j Represents sample x j The normalized eigenvalue of k Represents sample x k The normalized eigenvalue of , and k≠j, and δ is the preset scaling factor, fixed to 0.05; It represents the unlabeled set after each sample in the unlabeled set is subjected to two strong data enhancement transformations s_da(·) When actually calculating here, The second unlabeled set The unlabeled set after each sample in the strong data augmentation transformation s_da(·) is subjected to two The formula is as follows:

[0112]

[0113] In the formula, b represents the size of minibatch;

[0114] Finally, the overall loss L for backpropagation is total Calculated according to the following formula:

[0115] L total =L ssl +λ cl ·L d (8)

[0116] where λ cl is the weight coefficient, which is generally set to 0.025.

[0117] Step 1.34: The second DNN model is based on the first labeled set and the first unlabeled set First, the semi-supervised loss is calculated according to formula (5), and then for the first unlabeled set The contrast loss is calculated according to formula (6) to improve the feature extraction and clustering capabilities of the second DNN model, and the overall loss is back-propagated to update the parameters of the second DNN model; the back-propagation loss can be obtained according to formula (8).

[0118] Step 1.35, add 1 to the t value to determine whether the current iteration round t of the first overall training satisfies is the maximum total number of iterations. If it is satisfied, the next iteration of the first formal training will be carried out. If not, go to step 1.4.

[0119] Step 1.4: Select the last consecutive In the iterations of the rounds, all samples marked as True values ​​are put into the clean sample set (small clean set) D c , the formula is as follows:

[0120]

[0121] Where state[j][i] represents the marking state of the i-th sample of the j-th sequence of the first historical sequence s1.

[0122] Step 2: reinitialize the two DNN models, and perform a second pre-training and a second formal training on the two initialized DNN models based on a given initial noise label data set;

[0123] After obtaining the first DNN model and the second DNN model with preliminary discrimination ability through the second pre-training, in each round of iteration of the second formal training:

[0124] Using the first DNN model to label the samples of the initial noisy label data set, putting all samples indexed as True values ​​or belonging to the clean sample set in the labeling results of the first DNN model into the first labeled set, and putting the remaining samples of the initial noisy label data set into the first unlabeled set;

[0125] Re-label the samples of the initial noisy label data set using the second DNN model, put the samples whose index is marked as True or belongs to the clean sample set in the labeling result of the second DNN model into the second labeled set, and put the remaining samples of the initial noisy label data set into the second unlabeled set;

[0126] For the first DNN model, after semi-supervised and contrastive loss calculations are performed based on the second labeled set and the second unlabeled set, back-propagation is performed to update the parameters of the first DNN model;

[0127] For the second DNN model, after semi-supervised and contrastive loss calculations are performed based on the first labeled set and the first unlabeled set, back propagation is performed to update the parameters of the second DNN model;

[0128] After the training is completed, two final DNN models are obtained.

[0129] The process of step 2 is as follows Figure 2 As shown, specifically including:

[0130] Step 2.1: Preset the maximum total number of iterations And initialize the count of the current total iteration round t;

[0131] The maximum total number of iterations The target iteration round for the second pre-training and the sum of the target iteration rounds of the second formal training;

[0132] Step 2.2: Reinitialize the two DNN models using the GCE loss function based on the initial noisy label dataset The two DNN models are pre-trained for the second time, and the first DNN model and the second DNN model have preliminary discrimination capabilities through the second pre-training; wherein, x i represents the i-th image sample, Represents image sample x i The noisy labels, n is the number of samples in the initial noisy label dataset;

[0133] Step 2.3: After the second pre-training is completed, the two DNN models with preliminary discrimination capabilities are formally trained for the second time. Each iteration of the second formal training includes the following steps:

[0134] Step 2.31: For the first DNN model, calculate the initial noise label dataset The cross entropy loss of each sample in the two DNN models is used to estimate the loss distribution of the sample using a two-component GMM. The component with the smaller mean is selected as the estimated probability distribution of the sample containing a clean label, and the probability of the sample containing a clean label is expressed as w i , will satisfy w i The sample indexes with ≥τ are marked as True values, and the rest are marked as False, where τ is the preset threshold value;

[0135] Initialize the first labeled set and the first unlabeled set Only samples marked as True are put into the first labeled set The samples marked as True here include the clean samples screened in step 1, and then The indices of the remaining samples are marked as False and put into the first unlabeled set The specific formula is as follows:

[0136]

[0137] Step 2.32: For the second DNN model, calculate the cross entropy loss of each sample in the initial noise label data set on the two DNN models, use a two-component GMM to estimate the loss distribution of the sample, select the component with the smaller mean as the estimated probability distribution model of the sample containing the clean label, and determine the loss distribution probability w of each sample through the probability distribution model. i , for the sample indexes whose distribution probability is greater than or equal to the preset threshold, it is marked as True value, and the rest are marked as False value. In addition, for all the clean sample sets D obtained in step 1 c The sample index in is marked as True;

[0138] Initialize the second labeled set and the second unlabeled set After that, only the samples marked as True are put into the second labeled set The samples marked as True here include the clean samples screened in step 1, and the remaining samples marked as False are placed in the second unlabeled set. The specific formula is as follows:

[0139]

[0140]

[0141] Step 2.33: For the first DNN model, based on the second labeled set and the second unlabeled set According to formula (5), the semi-supervised loss is calculated for the second unlabeled set. The samples in the training set are used to calculate the contrast loss according to formula (6), and the overall loss is back-propagated to update the first DNN model parameters; at the same time, the present invention adopts oversampling technology in training to ensure that all unlabeled samples are learned at least once. The overall loss is calculated as follows:

[0142] L total =L ssl +λ cl ·L d (8) Where, L total is the overall loss, λ cl is the weight coefficient of contrast loss, L ssl is the semi-supervised loss, L cl is the contrast loss;

[0143] Step 2.34: For the second DNN model, based on the first labeled set and the first unlabeled set According to formula (5), the semi-supervised loss is calculated for the first unlabeled set The samples in are used to calculate the contrast loss according to formula (6), and the overall loss is back-propagated to update the second DNN model parameters; the overall loss is shown in formula (8). At the same time, the present invention adopts oversampling technology in training to ensure that all unlabeled samples are learned at least once.

[0144] Step 2.35, add 1 to the t value to determine whether the current iteration round t of the second overall training satisfies If it is satisfied, the next round of iteration of the second formal training will continue. If it is not satisfied, the training will be terminated, and two DNN models with strong classification performance and strong robustness to noise labels will be obtained.

[0145] Computer simulation results show that this method decouples the two objectives of sample selection and robustness training and optimizes them in steps. Combined with the introduced contrast loss, it can obtain better classification performance than similar methods and is suitable for all types of noise, including category-related noise, feature-related noise and real noise.

[0146] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A two-step noisy label sample learning method based on contrastive loss, characterized in that: include: Step 1: Select two DNN models with the same structure and define them as a first DNN model and a second DNN model; perform a first pre-training and a first formal training on the two DNN models based on a given initial noise label data set; the initial noise label data set consists of image samples and noisy labels of the image samples; After obtaining the first DNN model and the second DNN model with preliminary discrimination ability through the first pre-training, in the first formal training, the first DNN model and the second DNN model are used to screen clean samples from the initial noise label data set to construct a clean sample set, and the index of the clean sample is marked as a True value; step 1 specifically includes: Step 1.1: Preset the maximum total number of iterations And initialize the count of the current total iteration round t; The maximum total number of iterations The target iteration number for the first pre-training and the sum of the target iteration rounds of the first formal training; Step 1.2: Use the GCE loss function based on the given initial noise label dataset Performing a first pre-training on the two DNN models, so that the first DNN model and the second DNN model have preliminary discrimination capabilities through the first pre-training; Among them, x i represents the i-th image sample, Represents image sample x i The noisy labels, n is the number of samples in the initial noisy label dataset; At the end of each iteration of the first pre-training, the initial noise label dataset is calculated The cross entropy loss of the samples in the two DNN models is used to estimate the sample loss distribution using GMM. The samples whose distribution probability is greater than or equal to the preset threshold will have their indexes marked as True values. The indexes of the remaining samples are marked as False values, and the marking results are saved in the first historical sequence S1 and the second historical sequence S2 respectively according to the order of iteration rounds; Step 1.3: After the first pre-training is completed, the two DNN models with preliminary discrimination capabilities are formally trained for the first time. Each iteration of the first formal training includes the following steps: Step 1.31: For the first DNN model, first calculate the initial noise label dataset The cross entropy loss of each sample in the two DNN models is used to estimate the current loss distribution probability w of the sample using GMM. i ,Will Satisfy w i The indices of samples with ≥τ are marked as True values. The indexes of the remaining samples are marked as False values, τ is the preset threshold value, and then the marking results are saved in the first historical sequence S1 in the order of iteration rounds; based on the marking results in the current first historical sequence S1, the initial noise label data set is Select samples that are continuously marked as True and put them into the first labeled set Will The remaining samples are placed in the first unlabeled set Step 1.32: For the second DNN model, first calculate the initial noise label dataset The cross entropy loss of each sample in the two DNN models is used to estimate the current loss distribution probability w of the sample using GMM. i ,Will Satisfy w i The sample indices with ≥τ are marked as True values, The indexes of the remaining samples are marked as False, and τ is the preset threshold value; the marking results are saved in the second historical sequence S2 in the order of iteration rounds, and based on the marking results in the current second historical sequence S2, the initial noise label data set is Select samples that are continuously marked as True and put them into the second labeled set and will The remaining samples are placed in the second unlabeled set Step 1.33: For the first DNN model, based on the second labeled set and the second unlabeled set After calculating the semi-supervised loss, the second unlabeled set Compute the contrast loss calculation based on the samples in the above code, and propagate the overall loss back to update the first DNN model parameters; Step 1.34: The second DNN model is based on the first labeled set and the first unlabeled set After calculating the semi-supervised loss, the first unlabeled set Calculate the contrast loss and backpropagate the overall loss to update the second DNN model parameters; Step 1.35: Add 1 to the t value to determine whether the count of the current total iteration round t is less than or equal to If satisfied, proceed to the next iteration of the first formal training. If not satisfied, proceed to step 1.

4. Step 1.4: Select the last consecutive In the iterations of rounds, all samples marked as True values ​​are put into the clean sample set D c , the formula is as follows: In the formula, state[j][i] represents the marking state of the i-th sample of the j-th sequence of the first historical sequence S1; Step 2: reinitialize the two DNN models, and perform a second pre-training and a second formal training on the two initialized DNN models based on a given initial noise label data set; After obtaining the first DNN model and the second DNN model with preliminary discrimination ability through the second pre-training, in each round of iteration of the second formal training: Using the first DNN model to label the samples of the initial noisy label data set, putting all samples indexed as True values ​​or belonging to the clean sample set in the labeling results of the first DNN model into the first labeled set, and putting the remaining samples of the initial noisy label data set into the first unlabeled set; Re-label the samples of the initial noisy label data set using the second DNN model, put the samples whose index is marked as True or belongs to the clean sample set in the labeling result of the second DNN model into the second labeled set, and put the remaining samples of the initial noisy label data set into the second unlabeled set; For the first DNN model, after semi-supervised and contrastive loss calculations are performed based on the second labeled set and the second unlabeled set, back propagation is performed to update the parameters of the first DNN model; For the second DNN model, after semi-supervised and contrastive loss calculations are performed based on the first labeled set and the first unlabeled set, back-propagation is performed to update the parameters of the second DNN model; After the training is completed, two final DNN models are obtained.

2. The two-step noisy label sample learning method based on contrast loss as claimed in claim 1, characterized in that: Step 2 specifically includes: Step 2.1: Preset the maximum total number of iterations And initialize the count of the current total iteration round t; The maximum total number of iterations The target iteration round for the second pre-training and the sum of the target iteration rounds of the second formal training; Step 2.2: Reinitialize the two DNN models using the GCE loss function based on the initial noisy label dataset The two DNN models are pre-trained for the second time, and the first DNN model and the second DNN model have preliminary discrimination capabilities through the second pre-training; wherein, x i represents the i-th image sample, Represents image sample x i The noisy labels, n is the number of samples in the initial noisy label dataset; Step 2.3: After the second pre-training is completed, the two DNN models with preliminary discrimination capabilities are formally trained for the second time. Each iteration of the second formal training includes the following steps: Step 2.31: For the first DNN model, calculate the initial noise label dataset The cross entropy loss of each sample in the two DNN models is used, and GMM is used to estimate the current loss distribution probability w of the sample. i ,Will Satisfy w i The sample index ≥τ is marked as True value, τ is the preset threshold value; Initialize the first labeled set and the first unlabeled set After that, only the samples marked with True values ​​are put into the first labeled set The samples marked as True here include the clean samples screened in step 1, and then The indices of the remaining samples are marked as False values ​​and put into the first unlabeled set The specific formula is as follows: Step 2.32: For the second DNN model, calculate the cross entropy loss of each sample in the initial noise label data set on the two DNN models, and use GMM to estimate the current loss distribution probability w of the sample i ,Will Satisfy w i ≥τ sample indexes are marked as True values; Initialize the second labeled set and the second unlabeled set After that, only the samples marked as True are put into the second labeled set The samples marked as True here include the clean samples screened in step 1, and then The indices of the remaining samples are marked as False values ​​and put into the second unlabeled set The specific formula is as follows: Step 2.33: For the first DNN model, based on the second labeled set and the second unlabeled set After calculating the semi-supervised loss, the second unlabeled set Calculate the contrast loss for the samples in and backpropagate the overall loss to update the first DNN model parameters; The calculation formula of the overall loss is as follows: L total =L ssl +λ cl ·L cl Where, L total is the overall loss, λ cl is the weight coefficient of contrast loss, L ssl is the semi-supervised loss, L cl is the contrast loss; Step 2.34: For the second DNN model, based on the first labeled set and the first unlabeled set After calculating the semi-supervised loss, the first unlabeled set After calculating the contrast loss of the samples in, the overall loss is back-propagated to update the parameters of the second DNN model; Step 2.35: Add 1 to the t value to determine whether the current total iteration round t is less than or equal to If it is satisfied, the next round of iteration of the second formal training will be continued. If it is not satisfied, the training will be terminated to obtain two final DNN models.

3. The two-step noisy label sample learning method based on contrast loss as claimed in claim 1, characterized in that: In step 1.2, the GCE loss function is specifically: In the formula, l GCE is the GCE loss function, p(x i ) represents the classifier for the i-th image sample x i The predicted value of is the value after the softmax operation; The calculation formula of the cross entropy loss is as follows: In the formula, l ce represents the cross entropy loss.

4. The two-step noisy label sample learning method based on contrast loss as claimed in claim 1, characterized in that: In step 1.31, select the samples marked as True in the last K consecutive iterations in the first historical sequence S1 and put them into the first labeled set according to the following formula: Put all other samples into the first unlabeled set Wherein, state[j][i] represents the marking state of the i-th sample of the j-th sequence of the first historical sequence S1, and K is the number of times the preset sample is continuously marked as True.

5. The two-step noisy label sample learning method based on contrast loss as claimed in claim 1, characterized in that: In step 1.32, only the samples marked as True in the last K consecutive iterations in the second historical sequence S2 are selected and put into the second labeled set according to the following formula: Put all other samples into the second unlabeled set Wherein, state[j][i] represents the marking state of the i-th sample of the j-th sequence of the second historical sequence S2, and K is the number of times the preset sample is continuously marked as True.

6. The two-step noisy label sample learning method based on contrast loss as claimed in claim 1, characterized in that: Step 1.33 is as follows: For the first DNN model, the SSL loss function is used based on the second labeled set and the second unlabeled set The semi-supervised loss L is calculated according to the following formula ssl ; Among them, l x is the classification loss function, l u is the absolute mean loss, l reg is the regularization term, λ u and λ reg is the preset weight coefficient; for the second unlabeled set For each sample in , the contrast loss is calculated according to the following formula: Among them, L cl represents the contrast loss, l(i,j) represents the similarity value between the i-th sample and the j-th sample, z i Represents sample x i The normalized eigenvalue of j Represents sample x j The normalized eigenvalue of k Represents sample x k The normalized eigenvalue of , and k≠j, and δ is the preset scaling factor; and The second unlabeled set The unlabeled set after each sample in the strong data enhancement transformation s_da(·) is twice is expressed as follows: In the formula, b represents the size of minibatch; Finally, the overall loss L for backpropagation is total Calculated according to the following formula: L total =L ssl +λ cl ·L cl where λ cl is the weight coefficient.

7. The two-step noisy label sample learning method based on contrast loss according to claim 1 or 2, characterized in that: Use GMM to estimate the current loss distribution probability w of the sample i The method is: Use a two-component GMM to estimate the loss distribution of the sample, select the component with the smaller mean as the estimated probability distribution model of the sample containing the clean label, and determine the loss distribution probability w of each sample through the probability distribution model i .