Robust noisy label learning method based on double-flow sample distillation and related device

Abstract

The application discloses a robust noise label learning method based on double-flow sample distillation and a related device, belongs to the technical field of computer vision, takes image data and corresponding noise labels thereof as training samples, extracts features of the training samples, then performs sample division in a loss space and sample division in a feature space, obtains a certain set and an uncertain set, wherein the certain set comprises a first clean label sample set and a first noise label sample set; trains a meta-classification network according to the certain set, divides the uncertain set into a second clean label sample set and a second noise label sample set through the meta-classification network, obtains a labeled set in combination with the first clean label sample set and the second clean label sample set, obtains an unlabeled set in combination with the first noise label sample set and the second noise label sample set, and finally trains a target model by using a semi-supervised learning algorithm. The application can solve the problem of low robustness of noise label learning in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a robust noise label learning method and related apparatus based on two-stream sample distillation. Background Technology

[0002] Noisy label learning is an important and challenging task in deep learning, especially in image classification. It focuses on training robust models on large datasets with noisy labels. The importance of this problem stems from the imperfections in real-world labeled data, such as incorrect, missing, or ambiguous labels, which can significantly impact the performance of deep learning models.

[0003] Existing methods for learning noisy labels mainly focus on two aspects: sample selection and label correction. Sample selection methods train the model by identifying "clean" samples in the dataset, such as using small loss criteria or feature space-based clustering algorithms. Label correction methods, on the other hand, attempt to correct erroneous labels by estimating the noise distribution, such as using data augmentation or relabeling techniques. Although existing techniques have made some progress in handling noisy labels, several key issues and challenges remain. One is the limitation of sample selection: existing methods rely too heavily on loss values ​​or feature similarity, ignoring the complexity and diversity of noisy labels. Another is the instability of label correction: label correction based on the estimated noise distribution introduces additional uncertainty, affecting the model's generalization ability.

[0004] In summary, existing noise label learning methods still need improvement in enhancing model robustness, especially when dealing with complex noise patterns. Therefore, there is an urgent need to develop a new and more robust noise label learning method to address the shortcomings of current technologies. Summary of the Invention

[0005] The purpose of this invention is to provide a robust noise label learning method and related apparatus based on dual-stream sample distillation, so as to solve the problem of low robustness of existing noise label learning techniques.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] Firstly, a robust noise label learning method based on two-stream sample distillation includes the following steps:

[0008] Obtain image data and its corresponding noise labels;

[0009] Using the image data and its corresponding noise labels as training samples, the features of the training samples are extracted.

[0010] Based on the features, the training samples are divided into a loss space and a feature space to obtain a deterministic set and an uncertain set, wherein the deterministic set includes a first clean label sample set and a first noisy label sample set;

[0011] The determined set is used as metadata to train a meta-classification network. The meta-classification network divides the uncertain set into a second clean label sample set and a second noise label sample set. The first clean label sample set and the second clean label sample set are combined to obtain a clean set. The first noise label sample set and the second noise label sample set are combined to obtain a noise set.

[0012] The clean set is used as the labeled set, and the noise set is used as the unlabeled set. Based on the labeled set and the unlabeled set, a target model is trained using a semi-supervised learning algorithm to achieve robust noise label learning.

[0013] In some implementations, a ResNet model is used as the backbone network, and a convolutional neural network is employed to extract features from the training samples.

[0014] In some implementations, the step of partitioning the training samples into a loss space based on the features specifically includes:

[0015] The training samples are divided into multiple noise clusters based on the features, and each noise cluster contains multiple samples.

[0016] Calculate the cross-entropy loss for each sample in each noise cluster;

[0017] After setting a first threshold, the cross-entropy loss is used to divide the loss space into positive and negative sample sets using a Gaussian mixture model.

[0018] In some implementations, the step of partitioning the training samples into a feature space based on the features specifically includes:

[0019] After calculating the category center of each noise cluster, the cosine similarity between each sample and its corresponding category center is calculated based on the category center.

[0020] After setting a second threshold, the cosine similarity is used to divide the feature space into positive and negative sample sets using a Gaussian mixture model, resulting in a positive sample set and a negative sample set in the feature space.

[0021] In some implementations, the step of partitioning the training samples into a loss space and a feature space based on the features to obtain a deterministic set and an uncertain set specifically includes:

[0022] The overlapping portions of the positive sample set of the loss space and the positive sample set of the feature space, as well as the overlapping portions of the negative sample set of the loss space and the negative sample set of the feature space, are taken as the determined set, and the remaining portions are taken as the uncertain set.

[0023] The overlapping portion of the positive sample set in the loss space and the positive sample set in the feature space constitutes the first clean label sample set.

[0024] The overlapping portion of the negative sample set in the loss space and the negative sample set in the feature space constitutes the first noise label sample set.

[0025] In some implementations, the step of using the determined set as metadata to train a meta-classification network specifically includes:

[0026] After dividing the metadata into binary labels, the posterior probabilities in the feature space and loss space of the metadata are used as two-dimensional scores. The two-dimensional scores are then mapped to one-dimensional scores to obtain the meta-classification network.

[0027] In some implementations, the step of dividing the uncertain set into a second clean-label sample set and a second noisy-label sample set using the meta-classification network specifically includes:

[0028] A preset classification threshold is set, and the uncertain set is divided into a second clean label sample set and a second noisy label sample set according to the classification threshold and through the meta-classification network.

[0029] Secondly, a robust noise label learning system based on dual-stream sample distillation includes:

[0030] The data acquisition module is used to acquire image data and its corresponding noise labels;

[0031] The feature extraction module is used to extract features from the training samples using the image data and its corresponding noise labels as training samples.

[0032] The parallel sample segmentation module is used to perform sample partitioning of the loss space and sample partitioning of the feature space on the training samples according to the features, to obtain a deterministic set and an uncertain set, wherein the deterministic set includes a first clean label sample set and a first noisy label sample set;

[0033] The meta-sample purification module is used to train a meta-classification network using the determined set as metadata, and to divide the uncertain set into a second clean label sample set and a second noise label sample set through the meta-classification network. The first clean label sample set and the second clean label sample set are combined to obtain a clean set, and the first noise label sample set and the second noise label sample set are combined to obtain a noise set.

[0034] The model training module is used to take the clean set as the labeled set and the noise set as the unlabeled set, and train the target model using the labeled set and the unlabeled set and a semi-supervised learning algorithm.

[0035] Thirdly, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable in the processor, wherein the processor, when executing the computer program, implements the steps of the robust noise label learning method based on dual-stream sample distillation.

[0036] Fourthly, a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the robust noise label learning method based on dual-stream sample distillation.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] This invention provides a robust noise label learning method based on two-stream sample distillation. By simultaneously considering sample partitioning in both the feature space and loss space based on the extracted training sample features, it can more accurately identify clean samples in the dataset, providing more reliable data support for model training. The training samples are divided into a deterministic set and an uncertain set through sample partitioning in both the loss space and feature space. A meta-classification network is trained using the deterministic set, and then the uncertain set is further partitioned using the meta-classification network to obtain a clean set and a noise set. This method enables the target model to exhibit better robustness when facing noisy labels, reducing erroneous learning caused by noise labels. Furthermore, this robust noise label learning method is not dependent on a specific noise model and can adapt to different noise environments and data distributions, exhibiting good versatility. Therefore, this invention significantly improves the training effect and robustness of deep learning models on noisy datasets, solving the problem of low robustness in existing noise label learning techniques.

[0039] Furthermore, this invention employs a Gaussian mixture model and performs sample partitioning based on the cross-entropy loss of samples in the loss space and the cosine similarity of samples in the feature space. By jointly considering the sample structure in the feature space and human priors in the loss space, a highly reliable positive and negative sample training set can be generated, providing a reliable data foundation for subsequent model training.

[0040] Furthermore, after dividing the metadata into binary labels, the present invention uses the posterior probabilities in the feature space and loss space of the metadata as two-dimensional scores. After mapping the two-dimensional scores to one-dimensional scores, a meta-classification network is obtained. The meta-classification network is used to clean up the uncertain set, and more semi-hard samples with potentially clean labels are mined, thereby improving the quality of training samples and enhancing the efficiency and effectiveness of subsequent target model training. Mapping the two-dimensional scores to one-dimensional scores can more accurately evaluate the label quality of the samples. Attached Figure Description

[0041] Figure 1 The flowchart of the robust noise label learning method based on dual-stream sample distillation provided in this embodiment is shown.

[0042] Figure 2 This diagram illustrates the principle of grouping data features using causal inference theory in this embodiment.

[0043] Figure 3 This is a structural diagram of the meta-sample purification module in this embodiment;

[0044] Figure 4 This is a technical framework diagram of the robust noise label learning method based on Two-Stream Sample Distillation (TSSD) provided in this embodiment;

[0045] Figure 5 This is a diagram illustrating the effect of the meta-sample purification module in this embodiment. Figure 5 (a) Shows the improvement in data cleanliness achieved by the meta-sample cleansing module in the CIFAR-10 dataset. Figure 5 (b) The improvement in data cleanliness achieved by the metasample cleansing module in the CIFAR-100 dataset;

[0046] Figure 6 In this embodiment, the Clothing1m samples in the visualization section are divided into posterior probability differences using a Gaussian mixture module.

[0047] Figure 7 This is a schematic diagram of the robust noise label learning system based on dual-stream sample distillation provided in this embodiment;

[0048] Figure 8 This is a flowchart illustrating the robust noise label learning method based on dual-stream sample distillation provided by the present invention. Detailed Implementation

[0049] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings. The content described herein is for explanation rather than limitation of the present invention.

[0050] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification and claims of this invention are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, systems, products, or devices.

[0051] like Figure 1 and Figure 8 As shown, this embodiment provides a robust noise label learning method based on two-stream sample distillation, including the following steps:

[0052] Step 1: Acquire image data by capturing images from public datasets, professional databases, user-uploaded content, or specific scenarios;

[0053] Step 2: Preprocess the data and obtain noisy labels. In research, to evaluate the model's robustness to noise, noisy labels may be intentionally introduced into clean datasets. In the real world, labels for noisy labeled datasets may come from a variety of different sources and situations. In some datasets, labels may be annotated by non-professionals or people with limited training, which may lead to labeling errors or inconsistencies; some datasets may be initially labeled using automated tools or algorithms, which may not accurately identify all categories, especially in complex or ambiguous cases.

[0054] Step 2.1: Clean datasets used for research need to be manually noise-added; this step can be omitted for datasets that are already noisy.

[0055] Artificial noise addition specifically includes:

[0056] Step 2.1.1: Determine the noise model. First, determine the type of noise to be simulated. Noise types include random noise, symmetrical noise, asymmetrical noise, or noise models based on specific scenarios.

[0057] Step 2.2.2: Noise injection. Based on the selected noise type, the clean labels are noise-added. For example, random noise might involve randomly assigning labels to other classes for a subset of samples; symmetric noise might involve incorrectly assigning labels to all other classes with equal probability; asymmetric noise might involve label replacement based on inter-class similarity.

[0058] Step 2.2.3: Control the intensity or proportion of noise, which can be fixed or varied according to category or sample.

[0059] Step 2.2: Standardize image format and resolution. Convert all images to a uniform file format (such as JPEG or PNG) to ensure compatibility and consistency. To avoid image distortion, scale the images while maintaining their aspect ratio to adjust the resolution of all images to meet the requirements of the model input. Ensure that all images have the same number of color channels (e.g., RGB three channels).

[0060] Step 2.3: Perform data augmentation by applying data augmentation techniques such as rotation, flipping, scaling, and cropping to increase the diversity of the dataset and improve the generalization ability of the model. This step requires selecting appropriate data augmentation methods based on the characteristics of the dataset and the requirements of the model.

[0061] Step 2.4: Record the image's metadata, including the image's original source, labels before and after noise addition, image size, color space, etc.; at the same time, create a unique identifier for each image to facilitate data tracking and management.

[0062] Step 2.5: Divide the dataset into training, validation, and test sets. Determine the proportion of each set to ensure that the model can be trained on a sufficient number of samples, while also having enough samples for validation and testing. This facilitates model training, hyperparameter tuning, and performance evaluation.

[0063] Step 3: Feature extraction. A convolutional neural network is used as the feature extractor. The pre-trained model uses a residual network (ResNet) to extract the feature representation of each sample.

[0064] Specifically, a ResNet model is used as the backbone network, denoted as f, and a projection head h and a classification head g are added after it. The noise label dataset of the training set is denoted as:

[0065]

[0066] Will The data is input into a convolutional neural network to extract... The corresponding feature f(x).

[0067] Step 4: Parallel sample segmentation. The data is segmented using both loss space and feature space partitioning. In the loss space, the loss value for each sample is calculated, and a Gaussian mixture model is used to model the loss distribution. In the feature space, the similarity between a sample and its class center is calculated, and a Gaussian mixture model is also used to model the feature distribution. Based on the posterior probabilities of the loss and features, the samples are divided into initial clean sets and noise sets. For the overlapping portions of the noise and clean sets, these samples are classified as a deterministic set, while the remaining samples are classified as an uncertain set. Specifically:

[0068] In the simultaneous exploration of the loss space and feature space, the rationality of their collaborative use is first analyzed from the perspective of causal inference, such as... Figure 2 As shown, the input image set and noise label set All of these are factors that affect the network output f, where the input image set x is the noise label set. The determining factor, in practice, is whether the fluctuation of f is due to changes in the input image set χ or the noise label set. The changes are challenging; to determine the cause of the fluctuations in f, sample selection is crucial. Divide the K-class dataset into K noise clusters, i.e. in

[0069] This artificial intervention effectively Fixed as k i Thus, in x and response variable A clear and consistent causal relationship has been established between them, thus enabling the examination of x in each noise cluster. j and k i The individual impact on the prediction f, particularly analyzed from the perspectives of the loss space and feature space, is as follows:

[0070] On the one hand, it is assumed that the feature extractor is unbiased during training, so samples with clean labels in each noisy cluster will have similar responses, while samples with clean labels will have different responses from samples with noisy labels. Therefore, by exploring the sample structure in the feature space, it is possible to distinguish different types of samples.

[0071] On the other hand, it is further assumed that the classifier is unbiased during training, so that the prediction of a sample with a clean label will match its corresponding label, while the prediction of a sample with a noisy label will not match its corresponding label. By exploring human priors in the loss space, it is possible to distinguish different types of samples.

[0072] Based on the above analysis, parallel sample segmentation is performed. First, two sample distributions are modeled in the feature space and loss space, respectively, to represent the sample structure and human prior. Second, by performing sample clustering in the feature space and loss space, the training samples are refined into a deterministic set and an uncertain set.

[0073] Step 4.1: In the loss space, model the sample distribution by exploring the difference between the network prediction and the given noise label. The goal is to partition the samples by finding the optimal criterion from the resulting loss distribution. Specifically, for each noise cluster... Each sample (x) j ,k i The cross-entropy loss is calculated using the following formula:

[0074]

[0075] in It means (x) i ,k i Each x in ). j In practice, the small loss criterion is widely used for sample partitioning, where samples with a small loss criterion are considered to have clean labels, while samples with a large loss criterion are considered to have noisy labels.

[0076] Step 4.2: In the feature space, model the sample distribution by exploring the pairwise similarity differences among all samples within each noise cluster. The goal is to find an optimal criterion for sample partitioning from the resulting feature distribution. It is typically assumed that the pairwise similarity between samples within a class is much greater than the similarity between samples between classes. Therefore, explore the sample structure within each noise cluster. Specifically, first calculate the class center of each noise cluster using the following formula:

[0077]

[0078] Where, N c =|D i | represents the number of samples in each noise cluster. Then, the cosine similarity between each sample and its class center is calculated as follows:

[0079]

[0080] As a result, the entire training sample was divided into a deterministic set and an uncertain set by analyzing the cosine similarity difference between two samples with clean labels and the cosine similarity difference between one sample with a clean label and another sample with a noisy label.

[0081] To partition samples by analyzing the sample distribution in the loss space and feature space, a clustering algorithm is applied to divide the training samples in each noise cluster into a deterministic set and an uncertain set. Without loss of generality, we assume... and Since both follow a Gaussian mixture distribution, two Gaussian Mixture Models (GMMs) are used for sample partitioning, as shown in the following formula:

[0082]

[0083] in, and Representing samples (x) j ,k iThe posterior probability of a positive sample in both the loss space and the feature space. Furthermore, two thresholds, t1 and t2, are used to filter out samples with lower confidence levels, defined as follows:

[0084]

[0085] and Let represent the sets of positive samples obtained in the loss space and feature space, respectively, whose quality can be significantly preserved during training. Similarly, the quality of negative samples can be preserved using the same method, defined as follows:

[0086]

[0087] and These represent the sets of negative samples obtained in the loss space and feature space, respectively.

[0088] To further improve the quality of positive and negative samples, positive and negative samples are combined in the loss space and feature space, as shown in the following formula:

[0089]

[0090] Therefore, the final definitions of definite and uncertain sets are as follows:

[0091]

[0092] in, To determine the set, It is an uncertain set.

[0093] Step 5: Meta-sample purification. A meta-classifier is trained using samples from the deterministic set. This classifier evaluates the label quality of samples in the uncertain set. By setting a threshold, the samples in the uncertain set are secondary-divided into a clean-label sample set and a noisy-label sample set. The result of the secondary division is combined with the noisy and clean sets from the deterministic set in the initial division to obtain the final clean and noisy sets.

[0094] Specifically, after parallel sample partitioning, the quality of samples in the deterministic set can be strongly guaranteed, but these samples perform poorly in optimizing the parameters of Deep Neural Networks (DNNs). Therefore, it is necessary to mine more valuable samples from the uncertain set to enhance the representational ability of DNNs. In practice, difficult samples are more important than simple samples in network training because current networks are capable enough to handle simple samples, but their ability to handle difficult samples is still insufficient. However, due to the limitations of network representational ability, directly mining difficult samples during training is very challenging. Therefore, the plan is to gradually mine semi-difficult samples through meta-sample purification in order to continuously enhance the network's representational ability in iterations. The meta-sample purification module mainly consists of two parts: sample purification modeling and meta-distribution mapping, which will be explained in the following paragraphs.

[0095] Sample Purification Modeling: The goal of this part is to identify and extract samples that perform poorly in the current network, i.e., semi-hard samples. By analyzing the sample distribution in the loss space and feature space, it is possible to determine which samples have predictions that are inconsistent with the labels or have low similarity to other samples; these samples may be semi-hard samples.

[0096] Meta-distribution mapping: After identifying semi-hard samples, it is necessary to further understand the dynamic changes of these samples during the network learning process. Meta-distribution mapping involves mapping the current state of these samples to a meta-distribution, which can guide how to adjust the network parameters or learning strategies to better handle these samples.

[0097] By using sample purification modeling and meta-distribution mapping, the network's ability to handle difficult samples can be gradually improved while maintaining high accuracy on simple samples. This gradual purification and enhancement process helps the network to learn and predict more robustly and accurately when facing various complex situations.

[0098] The main problem with meta-sample purification lies in how to extract samples from uncertain sets. To extract valuable semi-hard samples, one direct approach is to adjust the posterior probability to... and Consider it as a two-dimensional fraction Then, a suitable model is designed to learn an appropriate splitting criterion, which can be simply stated as follows:

[0099]

[0100] in Let M(·) represent a one-dimensional score, which can be further used to mine semi-hard samples through additional threshold filtering. Furthermore, M(·) denotes a mapping function that transforms the two-dimensional score into a one-dimensional score. In practice, the simplest mapping function is a weighted average of two probabilities, defined as follows:

[0101]

[0102] Here, λ is a constant weight. This form of the mapping function only considers the linear relationship between two continuous probabilities and cannot simulate nonlinear relationships in some complex cases. Worse still, choosing a suitable weight λ is a very challenging problem in practice, which in turn degrades the performance of semi-hard sample mining. To address these challenges, a complete meta-distribution mapping solution is proposed, which can learn an optimal mapping function to solve the semi-hard sample mining problem.

[0103] Previous meta-learning strategies typically employ a separate set of clean-labeled samples as metadata during training, then learn a mapping function for sample cleansing. On one hand, determining the sample labels in the set is highly accurate because the sample structure in the feature space and human priors in the loss space have been explored during parallel sample partitioning; on the other hand, the metadata distribution is identical to that of the training data, as they are mined directly from the original dataset.

[0104] Once the metadata is ready, model training proceeds, with further fine-tuning of a meta-network m to align with the nonlinear mapping function. According to the general approximation theorem, in Figure 3 The system employs a multi-layer perceptron (MLP) to obtain the optimal nonlinear mapping function. Specifically, the network receives two-dimensional fractional... As input, they are then mapped to one-dimensional fractions. Furthermore, additional binary labels are assigned to these meta-samples during training. For the determined set... Its binary label b n The definition is as follows:

[0105]

[0106] Then, this metadata is used to calculate the binary cross-entropy loss, along with the predicted one-dimensional score. as follows:

[0107]

[0108] Given the above configuration, the optimization task can be defined as:

[0109]

[0110] Where Θ represents the optimization parameters. As a common practice, the well-known Stochastic Gradient Descent (SGD) algorithm is used to minimize the loss during training. An approximate mapping function is fitted to M≈m. * (Θ). Furthermore, the definition for converting a two-dimensional fraction to a one-dimensional fraction is estimated as follows:

[0111]

[0112] The higher the value, the greater the probability that the relevant sample has an accurate label. For Final sample purification can be achieved by setting two thresholds, t3 and t4. To execute, it can be described as follows:

[0113]

[0114] Where C u and These represent samples with clean labels and samples with noisy labels, respectively. This indicates the number of samples in the uncertain set. Ultimately, the entire dataset is divided into two parts: the dataset containing the accurate labels. and datasets containing error labels in and

[0115] Step 6: Semi-supervised learning training. Clean and concentrated samples are used as labeled data, and noisy and concentrated samples are used as unlabeled data. The model is trained using a semi-supervised learning algorithm.

[0116] Specifically, following the DivideMix method, clean datasets containing accurate labels are used. Specify as labeled dataset Simultaneously, the dataset containing error labels will be included. Labels are omitted, treating it as an unlabeled dataset. To train the network. For labeled samples According to probability The average prediction result of the mutual teaching network i n Adjust the original label As shown below:

[0117]

[0118] in This represents the refined label. For unlabeled samples, the average predictions from the mutual teaching network are used to "co-guess" the label q. nAfter the above operations, the expanded dataset was obtained. and

[0119] Next, apply the MixMatch method. and Convert to C ′ and U ′ .

[0120] The loss on C' is the cross-entropy loss, and the loss on U... ′ The loss is the mean squared error:

[0121]

[0122]

[0123] To prevent all samples from being assigned to a single class, a regularization term is further applied during training, defined as follows:

[0124]

[0125] Finally, the total loss can be expressed as follows:

[0126]

[0127] Where λ u and λ r This represents two constant weights.

[0128] The following section presents a qualitative and quantitative comparative experimental analysis of the robust noise label learning method proposed in this embodiment with existing methods. The effectiveness of the proposed method is verified on four public datasets: CIFAR-10, CIFAR-100, Tiny-ImageNet, and Clothing1m. Tables 1, 2, 3, and 4 show the quantitative experimental results of the proposed method. Tables 1 and 2 show the experimental results on the CIFAR-10 / 100 dataset, Table 3 shows the experimental results on the Tiny-ImageNet dataset, and Table 4 shows the experimental results on the Clothing1m dataset.

[0129] Table 1 shows the experimental results on the CIFAR-10 dataset.

[0130]

[0131] Table 2 shows the experimental results on the CIFAR-100 dataset.

[0132]

[0133] Table 3 shows the experimental results on the Tiny-ImageNet dataset.

[0134] Methods Tiny-ImageNet 0% Tiny-ImageNet 20% Tiny-ImageNet 50% CE 57.4％ 35.8％ 19.8％ Decoupling - 37.0％ 22.8％ F-correction - 44.5％ 33.1％ MentorNet - 45.7％ 35.8％ Co-teaching+ 52.4％ 48.2％ 41.8％ M-correction 57.7％ 57.2％ 51.6％ NCT 62.4％ 58.0％ 47.8％ UNICON 62.7％ 59.2％ 52.7％ TSSD 63.1％ 60.9％ 53.5％

[0135] Table 4 shows the experimental results on the Clothing1m dataset.

[0136] Methods Backbone Accuracy CE ResNet-50 69.2％ Joint-Optim ResNet-50 72.0％ MetaCleaner ResNet-50 72.5％ PCIL ResNet-50 73.5％ DivideMix ResNet-50 74.8％ ELR ResNet-50 74.8％ UNICORN ResNet-50 74.9％ CC ResNet-50 75.4％ TCL ResNet-50 74.8％ OT-Filter ResNet-50 74.5％ TSSD ResNet-50 75.6％

[0137] As shown in the table, the bolded results are the optimal results. The robust noise label learning method (TSSD) based on dual-stream sample distillation proposed in this invention has achieved optimal performance on multiple datasets with different noise levels.

[0138] Tables 1 and 2 present the average performance on the CIFAR-10 and CIFAR-100 datasets, considering 20%, 50%, and 80% symmetric noise levels, and 10%, 30%, and 40% asymmetric noise levels. The proposed method in this embodiment outperforms most state-of-the-art methods, showing significant improvements, particularly under common noise conditions. However, the results of the proposed method are slightly inferior to those achieved by TCL on the CIFAR-100 dataset with 80% symmetric noise. This difference may be attributed to insufficient accuracy of the meta-samples selected from the feature space and loss space at higher noise levels, thus affecting the performance of the meta-classifier. As a potential solution, incorporating a small subset of clean data could be considered to mitigate this problem.

[0139] Table 3 shows the average performance on the Tiny-ImageNet dataset at 0%, 20%, and 50% symmetric noise levels. The proposed method in this embodiment outperforms most state-of-the-art methods and follows the same training methodology as UNICON. Unlike UNICON, which only uses JS divergence for sample selection in the loss space, the proposed method also includes sample selection in the feature space. This allows the proposed method to identify more potentially clean samples and improves the accuracy of clean samples using the PSD module. Therefore, we observed a performance improvement of approximately 1%.

[0140] Table 4 shows the average performance on the Clothing-1M dataset in real-world scenarios, demonstrating that the proposed method outperforms state-of-the-art methods and achieves excellent results. The proposed method uses the same training method as CC, but introduces a loss space component, unlike CC which only uses the feature space. Furthermore, the MSP module proposed in this embodiment integrates both spaces, resulting in a performance improvement of approximately 0.2% compared to CC. These findings indicate that feature space filtering alone cannot identify challenging samples in the loss space. Including these challenging samples significantly contributes to the performance improvement.

[0141] Below are some qualitative explanations. Figure 4 This paper introduces a technical framework for a noise label learning system based on two-stream sample distillation. Figure 5 The results demonstrate how the metasample cleansing module improves data cleanliness in the CIFAR-10 and CIFAR-100 datasets. Figure 6 We visualized the posterior probability differences of some Clothing1m samples using GMM partitioning in a two-stream sample distillation module.

[0142] This embodiment also provides a robust noise label learning system based on dual-stream sample distillation, such as Figure 7 As shown, it includes:

[0143] The data acquisition module is used to acquire image data and its corresponding noise labels.

[0144] The feature extraction module is used to extract features from the training samples using the image data and its corresponding noise labels as training samples.

[0145] The parallel sample segmentation module is used to perform sample partitioning of the loss space and the feature space on the training samples according to the features, to obtain a deterministic set and an uncertain set, wherein the deterministic set includes a first clean label sample set and a first noisy label sample set; this module generates a training set with sufficiently reliable positive and negative samples by simultaneously considering the sample structure in the feature space and the human prior in the loss space.

[0146] The parallel sample segmentation module also includes sample distribution modeling and dual-space sample distillation. The sample distribution modeling is as follows: in the loss space, the sample distribution is modeled by calculating the cross-entropy loss of each sample to reflect the difference between the network prediction and the given noise label; in the feature space, the sample distribution is modeled by calculating the cosine similarity between the sample and the class center to reflect the similarity between the samples.

[0147] Dual-space sample distillation involves using a Gaussian mixture model to model the sample distribution in the loss space and feature space, and using two preset thresholds to filter out positive and negative samples with high confidence.

[0148] The meta-sample purification module is used to train a meta-classification network using the determined set as metadata. The meta-classification network divides the uncertain set into a second clean label sample set and a second noisy label sample set. The first clean label sample set and the second clean label sample set are combined to obtain a clean set, and the first noisy label sample set and the second noisy label sample set are combined to obtain a noise set. This module is further designed to mine sufficient semi-hard samples from the remaining uncertain training set and learn a powerful meta-classifier by using additional gold data.

[0149] Specifically, the meta-sample purification module includes:

[0150] Sample purification modeling: Treat the posterior probabilities in the loss space and feature space as two-dimensional fractions, and design a mapping model to learn an appropriate splitting criterion;

[0151] Meta-distribution mapping: Using positive and negative samples from a defined set as metadata, a meta-network (Meta-Net) is trained that learns to map two-dimensional scores to one-dimensional scores.

[0152] Sample purification: By setting a classification threshold, one-dimensional scores are mapped to final sample labels, thereby filtering out samples with clean labels from the uncertain set.

[0153] The model training module is used to take the clean set as the labeled set and the noise set as the unlabeled set, and train the target model using the labeled set and the unlabeled set and a semi-supervised learning algorithm.

[0154] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0155] This embodiment also provides a computer device, which includes a processor and a memory. The memory is used to store a computer program (in this embodiment, the computer program includes computational components and iterative components, capable of model calculation and model updating). The computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to realize the corresponding method flow or corresponding function. The processor described in this embodiment can be used for the operation of a robust noise label learning method based on dual-stream sample distillation.

[0156] This embodiment also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the robust noise label learning method based on dual-stream sample distillation in the above embodiment.

[0157] In summary, this embodiment discloses a robust noise label learning method and related apparatus based on two-stream sample distillation, which is used to improve model training performance under the influence of noise labels in deep learning. The method provided in this embodiment can effectively identify and utilize high-quality samples in the feature space and loss space, and further enhances the model's ability to handle semi-difficult samples through a meta-sample cleansing module, thereby significantly improving learning performance in noisy environments. Unlike other methods, the TSSD framework provided in this embodiment has the following characteristics:

[0158] (1) Dual-stream processing: Through the collaborative work of the parallel sample division module (PSD) and the meta sample purification module (MSP), this embodiment can evaluate and refine samples from two dimensions simultaneously, enhancing robustness to noise labels.

[0159] (2) High-quality sample extraction: The PSD module can generate a set of positive and negative samples with high confidence. These samples are considered to be representative of clean labels, providing a reliable data foundation for subsequent training.

[0160] (3) Semi-difficult sample mining: The MSP module learns a meta-classification network to conduct in-depth analysis of samples in the uncertain set and mine more semi-difficult samples with potential clean labels.

[0161] (4) Meta-learning application: By using a meta-classification network to learn the mapping from two-dimensional posterior probability to one-dimensional score, this invention can more accurately evaluate the label quality of samples.

[0162] (5) Experimental verification: Extensive experiments on the above-mentioned benchmark datasets have verified the effectiveness of the method provided in this embodiment, especially under different types and proportions of noise, it shows a significant performance improvement compared with the prior art.

[0163] (6) Wide applicability: The method provided in this embodiment is not only applicable to synthetic noise datasets, but also to noise labeling problems caused by diverse data sources in the real world, and has high practical application value.

[0164] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0166] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0167] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A robust noise label learning method based on two-stream sample distillation, characterized in that, Includes the following steps: Obtain image data and its corresponding noise labels; Using the image data and its corresponding noise labels as training samples, the features of the training samples are extracted. Based on the features, the training samples are divided into a loss space and a feature space to obtain a deterministic set and an uncertain set, wherein the deterministic set includes a first clean label sample set and a first noisy label sample set; The determined set is used as metadata to train a meta-classification network. The meta-classification network divides the uncertain set into a second clean label sample set and a second noise label sample set. The first clean label sample set and the second clean label sample set are combined to obtain a clean set. The first noise label sample set and the second noise label sample set are combined to obtain a noise set. The clean set is used as the labeled set, and the noise set is used as the unlabeled set. Based on the labeled set and the unlabeled set, a target model is trained using a semi-supervised learning algorithm to achieve robust noise label learning.

2. The robust noise label learning method based on two-stream sample distillation according to claim 1, characterized in that, The ResNet model was used as the backbone network, and a convolutional neural network was used to extract features from the training samples.

3. The robust noise label learning method based on two-stream sample distillation according to claim 1, characterized in that, The step of partitioning the training samples into a loss space based on the features specifically includes: The training samples are divided into multiple noise clusters based on the features, and each noise cluster contains multiple samples. Calculate the cross-entropy loss for each sample in each noise cluster; After setting a first threshold, the cross-entropy loss is used to divide the loss space into positive and negative sample sets using a Gaussian mixture model.

4. The robust noise label learning method based on dual-stream sample distillation according to claim 3, characterized in that, The step of partitioning the training samples into feature spaces based on the features specifically includes: After calculating the category center of each noise cluster, the cosine similarity between each sample and its corresponding category center is calculated based on the category center. After setting a second threshold, the cosine similarity is used to divide the feature space into positive and negative sample sets using a Gaussian mixture model, resulting in a positive sample set and a negative sample set in the feature space.

5. A robust noise label learning method based on two-stream sample distillation according to claim 4, characterized in that, The steps of partitioning the training samples into a loss space and a feature space based on the features to obtain a definite set and an uncertain set specifically include: The overlapping portions of the positive sample set of the loss space and the positive sample set of the feature space, as well as the overlapping portions of the negative sample set of the loss space and the negative sample set of the feature space, are taken as the determined set, and the remaining portions are taken as the uncertain set. The overlapping portion of the positive sample set in the loss space and the positive sample set in the feature space constitutes the first clean label sample set. The overlapping portion of the negative sample set in the loss space and the negative sample set in the feature space constitutes the first noise label sample set.

6. A robust noise label learning method based on two-stream sample distillation according to claim 5, characterized in that, The step of using the determined set as metadata to train a meta-classification network specifically includes: After dividing the metadata into binary labels, the posterior probabilities in the feature space and loss space of the metadata are used as two-dimensional scores. The two-dimensional scores are then mapped to one-dimensional scores to obtain the meta-classification network.

7. A robust noise label learning method based on two-stream sample distillation according to claim 6, characterized in that, The step of dividing the uncertain set into a second clean label sample set and a second noisy label sample set using the meta-classification network specifically includes: A preset classification threshold is set, and the uncertain set is divided into a second clean label sample set and a second noisy label sample set according to the classification threshold and through the meta-classification network.

8. A robust noise label learning system based on two-stream sample distillation, characterized in that, include: The data acquisition module is used to acquire image data and its corresponding noise labels; The feature extraction module is used to extract features from the training samples using the image data and its corresponding noise labels as training samples. The parallel sample segmentation module is used to perform sample partitioning of the loss space and sample partitioning of the feature space on the training samples according to the features, to obtain a deterministic set and an uncertain set, wherein the deterministic set includes a first clean label sample set and a first noisy label sample set; The meta-sample purification module is used to train a meta-classification network using the determined set as metadata, and to divide the uncertain set into a second clean label sample set and a second noise label sample set through the meta-classification network. The first clean label sample set and the second clean label sample set are combined to obtain a clean set, and the first noise label sample set and the second noise label sample set are combined to obtain a noise set. The model training module is used to take the clean set as the labeled set and the noise set as the unlabeled set, and train the target model using the labeled set and the unlabeled set and a semi-supervised learning algorithm.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable in the processor, wherein the processor executes the computer program to implement the steps of the robust noise label learning method based on dual-stream sample distillation as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the robust noise label learning method based on dual-stream sample distillation as described in any one of claims 1 to 7.

Images