Noise-free prototype guided representation calibration method capable of resisting label noise

Abstract

The invention discloses a label-noise-resistant noiseless prototype-guided representation calibration method in the technical field of machine learning and computer vision, and the method comprises the following steps: 1, a representation enhancement stage: obtaining enhanced depth representation through unsupervised contrast learning pre-training; the method comprises the following steps: 1, carrying out noise supervision on biased characterization, 2, carrying out characterization calibration, namely calibrating biased characterization under noise supervision by utilizing the noise-independent and fixed prototype obtained in the first stage, and 3, carrying out model training by combining cross entropy loss under noise supervision and characterization calibration loss established by analysis. According to the characterization calibration method, characterization learning under noise supervision is guided by using unsupervised priori knowledge, and the generalization ability of the model in a label noise scene is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of machine learning and computer vision, and in particular to a noise-free prototype-guided representation calibration method that is resistant to label noise. Background Art

[0002] Deep neural networks (DNNs) have been widely used in many fields and have demonstrated excellent performance, mainly due to large-scale, high-quality and accurately labeled datasets. However, in practice, obtaining accurately labeled large-scale datasets is not only costly but also time-consuming. In addition, due to the inaccuracy of manual labeling, crowdsourcing and automatic labeling algorithms, datasets in real-world scenarios are often inevitably contaminated by varying degrees of label noise. Although deep networks have powerful learning capabilities, they often lack the ability to distinguish between clean labels and noisy labels, which causes the model to eventually memorize and overfit the noisy labels, seriously affecting its classification performance and generalization ability. Therefore, label noise learning came into being and has become an important subfield in the field of weak supervision. It aims to use datasets containing noisy labels to design robust learning algorithms to achieve accurate classification of unseen test samples as much as possible. In recent years, existing label noise learning methods mainly include the following three categories:

[0003] The first category is robust loss functions. Since the traditional cross-entropy loss function (CE) has been proven to be difficult to handle in scenarios with label noise, researchers have proposed a variety of new loss functions to mitigate overfitting to mislabeled data in order to maintain risk consistency between learning with noisy labels and learning with clean labels. For example, the generalized cross-entropy loss (GCE) simultaneously leverages the noise robustness of the mean absolute error (MAE) and the fast convergence of the cross-entropy (CE), achieving a balance between the two. LogitClip constrains and defines the cross-entropy loss by clipping the logistic vector. The asymmetric no-hinge loss (AUL) improves on the no-hinge loss to improve the robustness of the loss. Wang et al. proposed the symmetric cross-entropy loss (Symmetric Loss), which, based on symmetry conditions, enhances the robustness of the cross-entropy loss by leveraging the noise robustness of the inverse cross-entropy. Although these robust loss functions generally have theoretical guarantees, they often rely on specific assumptions, and their generalization ability in complex real-world scenarios is still significantly limited.

[0004] The second category is loss correction methods. These methods adjust the training loss in various ways to counteract the negative effects of label noise. For example, Liu et al. proposed consistent loss reweighting, which redistributes data weights to optimize the training process. Goldberger et al. proposed adding an additional noise adaptation layer to the network, which connects clean labels with noisy labels to correct the network output. In addition, some studies estimate the transition probability matrix to reveal the transition relationship from clean labels to noisy labels, thereby recovering the posterior probability of the sample to correct the loss. However, the estimation of the transfer matrix usually depends on the anchor point. When the anchor point cannot be identified, the number of categories is large, or the noise situation is complex, the estimation of the noise transfer matrix is ​​not accurate.

[0005] The third category is sample selection methods, which use specific criteria to select potential clean samples from noisy datasets for model training and updating. Common criteria include the low-loss criterion based on the memory effect, which states that deep neural networks tend to memorize clean patterns first and then gradually memorize noise. A notable characteristic of this is that the training loss for clean samples is typically small, while the loss for noisy samples is generally large. In addition to the low-loss criterion, other methods include principles based on consistency or inconsistency of predictions. These methods typically require a known noise ratio and train two networks in parallel to reduce error accumulation. However, in practical applications, prior knowledge such as the noise ratio is often difficult to obtain. Furthermore, these methods lack unified sample selection criteria, and the criteria they use are overly empirical, lacking theoretical support and in-depth data mining. There is no guarantee that any single criterion will be applicable to all datasets. Furthermore, training two networks simultaneously not only significantly increases computational overhead but can also lead to unstable training processes.

[0006] Prototype-based methods for learning with noisy labels have also received considerable attention in recent years. These methods primarily use prototypes to achieve label correction and sample selection. A prototype is a vector that accurately represents the semantics of a category in the representation space, i.e., the class center. Due to its excellent generalization and robustness, prototypes have been extensively explored in previous research. Existing methods for obtaining prototypes include: generating an initial prototype after obtaining a relatively reliable feature extractor through noisy label preheating, selecting samples with high density as prototypes, or directly incorporating the prototype as a learnable component into the loss function. Most methods typically calculate the mean of the features of similar samples as the prototype and dynamically update the prototype during training. However, accurate prototype estimation relies on the dual correctness of the label and representation, which is difficult to ensure under conditions of label noise. In the presence of label noise, this method of relying on label information to construct prototypes may cause the prototype to deviate from its optimal position, making it difficult to effectively guide the model to learn stable and accurate classification boundaries. Summary of the Invention

[0007] The purpose of the present invention is to provide a noise-free prototype-guided representation calibration method that is resistant to label noise. It solves the technical defect of traditional methods that cause prototype offset due to reliance on noisy labels. It generates robust prototypes through unsupervised contrastive learning, and calibrates the representation under noisy labels by combining consistency constraints with a dynamic weighting mechanism. It does not need to rely on noisy labels to obtain robust unsupervised prior knowledge, significantly improving the generalization ability of the model in label noise scenarios, and is suitable for tasks such as image classification and target detection.

[0008] In order to achieve the above object, the technical solution of the present invention is as follows:

[0009] A noise-free prototype-guided characterization calibration method that is resistant to label noise comprises the following steps:

[0010] S1. Stage 1: Representation enhancement stage, obtaining enhanced deep representations through unsupervised contrastive learning pre-training;

[0011] S2. Stage 2: Representation calibration. Using the noise-independent and fixed prototypes obtained in the first stage, we calibrate the biased representations under noise supervision. By imposing directional constraints, we guide the representations affected by noise to the correct position in the feature space.

[0012] S3. Stage 3: Combine the cross entropy loss under noise supervision and the representation calibration loss established in the above analysis to train the model.

[0013] Furthermore, the S1 includes the following steps:

[0014] S11 adopts the MoCo framework, pre-trains the encoder network, and uses a momentum-based update mechanism to learn visual representations by comparing positive and negative sample pairs in a dynamic memory bank;

[0015] S12. Construct a contrastive loss using positive and negative sample pairs to optimize the feature extractor.

[0016] Furthermore, the S11 includes the following steps:

[0017] S111.MoCo framework for each input sample x i , generate its two augmented views: one is the query view x i q , the other is the key view x i k ;

[0018] S112. Set the query view x i q and key view x i kInput to two encoders with the same network structure but different initializations, and obtain low-dimensional embedding features z by mapping. i q and z i k+ , query view x i q and the corresponding key view x i k constitutes a positive sample pair, and the remaining key views in the current batch are used as negative samples.

[0019] Furthermore, the S12 includes the following steps:

[0020] S121. For input sample x i , the contrastive loss is defined as:

[0021]

[0022] where {k + , k -} represents the set of positive and negative bonds, and τ>0 is the temperature parameter;

[0023] S122. Minimize the loss function shown in the contrast loss formula to train an enhanced contrast representation;

[0024] S123. Keep the query encoder f′(·) with parameter θ0 for subsequent initial feature extraction.

[0025] Furthermore, the S2 includes the following steps:

[0026] S21. Construction and assignment of noise-free prototypes: Assign a prototype to each instance to generate a set of category prototypes. For each instance, assign the category prototype closest to it to form a fixed and noise-independent prototype set corresponding to all samples.

[0027] S22. Explicitly calibrating biased representations under noisy conditions via consistency regularization constraints and loss-based dynamic regularization constraints.

[0028] Furthermore, the S21 includes the following steps:

[0029] S211. Through pre-trained encoder Extract unsupervised representations for all training data:

[0030]

[0031] in, Represents sample x i Corresponding unsupervised deep representation;

[0032] S212. After extracting the unsupervised deep representation, perform k-means clustering on Z and record the prototypes representing each category k∈[K]:

[0033]

[0034] Where K is the total number of categories, c is the prototype set representing all categories, and based on this prototype set, the Euclidean distance between the unsupervised representation of each sample and all prototypes is calculated as follows:

[0035]

[0036] S213. For each instance, assign the category prototype closest to it to form a prototype set corresponding to all samples

[0037]

[0038] Furthermore, the S22 includes the following steps:

[0039] S221. Consistency Regularization Constraint: Under noisy supervision, use a pre-trained feature extractor to initialize the representation and optimize the feature extractor and classifier simultaneously. For classification tasks, add a linear head and use the following cross-entropy loss:

[0040]

[0041] in, Noise label One-hot encoding of Providing noisy but valuable supervisory information, together with the prior knowledge provided by the unsupervised prototype, guides representation learning. The following distance constraints guide the subsequent learned representation to be consistent with its assigned prototype C:

[0042]

[0043] S222. Loss-based dynamic regularization constraint: Use the size of the loss value to impose dynamic representation calibration constraints on each sample, that is, for each training sample We use the cross entropy loss value calculated in the current training cycle As the weighting coefficient, the following loss-based adaptive regularization term is constructed:

[0044]

[0045] Furthermore, the S3 includes the following steps:

[0046] S31. Construct the overall objective function as follows:

[0047]

[0048] Where λ1 and λ2 are hyperparameters used to control the strength of representation calibration. λ1 is always kept greater than zero to ensure stable and invariant representation calibration. In complex noise scenarios, λ2 is set to be greater than zero to provide dynamic personalized constraints. The second and third terms of the above loss function are combined to express as:

[0049]

[0050] S32. Perform model training based on the overall objective function.

[0051] By adopting the above technical solution, the present invention has the following advantages:

[0052] The present invention provides a noise-free prototype-guided representation calibration method that is resistant to label noise. Prototypes are generated through unsupervised learning, and prototype construction is completely independent of noise label supervision, completely avoiding the interference of noise on the prototype. The obtained prototype is fixed to guide representation learning, significantly improving the robustness of representation, and effectively overcoming the technical defects of traditional methods that cause prototype offset due to reliance on noise labels.

[0053] Compared with the existing end-to-end training methods, this paper focuses on the representation behavior under label noise, performs label noise robust learning from the perspective of explicit representation calibration, and pioneers a representation calibration framework that combines consistency regularization constraints with loss-based adaptive dynamic constraints to achieve effective representation recovery and robust model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Flowchart of the noise-free prototype-guided characterization calibration method for robustness against label noise. DETAILED DESCRIPTION

[0055] The technical solution of the present invention is described in detail below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as first and second, etc., are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include," "comprise," or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or apparatus.

[0056] A noise-free prototype-guided characterization calibration method that is resistant to label noise is specifically as follows Figure 1 As shown, the following steps are included:

[0057] S1. Stage 1: Representation enhancement stage, in which enhanced deep representations are obtained through unsupervised contrastive learning pre-training. This stage is completely independent of label supervision, ensuring that the obtained representation is not contaminated by label noise, laying the foundation for subsequent processing.

[0058] In recent years, unsupervised contrastive learning has demonstrated significant potential for representation learning in a variety of fields, including computer vision and natural language processing. As a specialized unsupervised learning method, contrastive learning can learn discriminative embedding representations without manual annotation. Because this process is completely independent of the labels of the training data and can capture the essential structure of the data, the learned representations are unaffected by incorrect labels and possess inherent robustness. Based on this, the present invention utilizes deep representations obtained through contrastive learning to enhance representation learning in the presence of label noise.

[0059] Specifically, contrastive learning obtains robust representations by maximizing the similarity between visually similar samples while minimizing the similarity between unrelated samples. This mechanism enables the model to tightly cluster similar instances in the latent embedding space while effectively separating dissimilar instances.

[0060] S1 includes the following steps:

[0061] S11 uses the MoCo framework to pre-train the encoder network. The MoCo framework uses a momentum-based update mechanism to learn visual representations by comparing positive and negative sample pairs in a dynamic memory bank.

[0062] S11 includes the following steps:

[0063] S111.MoCo framework for each input sample x i , generate its two augmented views: one is the query view x i q , the other is the key view x i k ;

[0064] S112. Set the query view x i q and key view x i k Input to two encoders with the same network structure but different initializations, and obtain low-dimensional embedding features z by mapping. i q and z i k+ , query view x i q and the corresponding key view x i k constitutes a positive sample pair, and the remaining key views in the current batch are used as negative samples.

[0065] S12. Construct a contrastive loss using positive and negative sample pairs to optimize the feature extractor.

[0066] S12 includes the following steps:

[0067] S121. For input sample x i , the contrastive loss is defined as:

[0068]

[0069] where {k + , k -} represents the set of positive and negative bonds, τ>0 is the temperature parameter;

[0070] S122. By minimizing the loss function shown in formula (1), an enhanced contrast representation can be trained.

[0071] S123. Finally, we retain the query encoder f′(·) with parameter θ0 for subsequent initial feature extraction. In addition, this feature extractor will serve as the initial model for subsequent training, significantly accelerating the convergence of the model.

[0072] S2. Stage 2: Representation calibration. Using the noise-independent and fixed prototypes obtained in the first stage, we calibrate the biased representations under noise supervision. By imposing directional constraints, we guide the representations affected by noise to the correct position in the feature space.

[0073] Similar to the study by Zhang et al., by decoupling noisy label training into two stages: representation learning and classifier learning, we found that representations (feature extractors) are more susceptible to label noise than classifiers. In essence, the representations of mislabeled samples tend to cluster in the direction of their corresponding incorrect labels, gradually deviating from their true cluster centers. This shows that label noise mainly reduces the classification performance of the model by destroying the clustering structure of the representation. In addition, we further observed that when the learned representations can be tightly clustered, the classifier can still establish a relatively accurate decision boundary even in the presence of label noise. Based on this discovery, the present invention focuses on suppressing the representation's memory of mislabeled data and restoring the representation structure to a state that is relatively less contaminated by noise.

[0074] Among them, S2 includes the following specific steps:

[0075] S21. Construction and assignment of noise-free prototypes: Assign a prototype to each instance to generate a set of category prototypes. For each instance, assign the category prototype closest to it to form a fixed and noise-independent prototype set corresponding to all samples.

[0076] S21 includes the following steps:

[0077] S211. Through pre-trained encoder Extract unsupervised representations for all training data:

[0078]

[0079] in, Represents sample x i Corresponding unsupervised deep representation;

[0080] S212. After extracting the representation, the key is how to fully extract and mine the effective information contained in the unsupervised deep representation. As mentioned above, the present invention uses prototypes to carry the valuable information in these representations to solve the label noise problem. To this end, k-means clustering is performed on Z, and the prototypes representing each category are recorded. k∈[K]:

[0081]

[0082] Where K is the total number of categories, c is the prototype set representing all categories, and based on this prototype set, the Euclidean distance between the unsupervised representation of each sample and all prototypes is calculated as follows:

[0083]

[0084] S213. For each instance, assign the category prototype closest to it to form a prototype set corresponding to all samples

[0085]

[0086] The prototype obtained by the above unsupervised method avoids access to label information and minimizes label noise interference, so it has strong robustness and provides a reliable basis for subsequent characterization calibration.

[0087] S22. Explicitly calibrating biased representations under noisy conditions via consistency regularization constraints and loss-based dynamic regularization constraints.

[0088] S22 includes the following steps:

[0089] S221. Consistency Regularization Constraint: Under noisy supervision, use a pre-trained feature extractor to initialize the representation and optimize the feature extractor and classifier simultaneously. For classification tasks, add a linear head and use the following cross-entropy loss:

[0090]

[0091] in, Noise label One-hot encoding of Providing noisy but valuable supervisory information, together with the prior knowledge provided by the unsupervised prototype, guides representation learning. The present invention uses the noise-free prototype as a fixed anchor point, which helps to ensure stable and consistent representation learning guidance and enhance the robustness of the representation. Technically, the present invention guides the subsequent learned representation to be consistent with its assigned prototype C through the following distance constraint:

[0092]

[0093] The noise-free prototype-guided distance constraint effectively brings biased representations affected by label noise back to a relatively accurate position. By applying consistent and time-invariant weights to each sample, we ensure stable guidance for representation learning in the presence of label noise.

[0094] S222. Loss-based dynamic regularization constraints: Although consistency regularization constraints can effectively calibrate representations, personalized strategies still need to be designed to cope with complex high-noise scenarios. The memory effect of deep neural networks (DNNs) shows that DNNs will prioritize learning simple patterns and then gradually fit noise patterns, which makes clean samples usually show smaller loss values ​​than noisy samples (the core idea of ​​the small loss criterion). Based on this, we found that the cross-entropy loss value generated during training can not only assist in distinguishing clean samples from noisy samples, but also serve as a natural weight and gradually decay as the training progresses. It is worth noting that some clean difficult-to-learn samples may exhibit similar training dynamics to noisy data, and therefore also show higher loss values. Such samples have potential value for improving the generalization ability of the model, so they should also be given higher weights during training to promote their more effective representation learning. Inspired by this, the size of the loss value is used to impose a dynamic representation calibration constraint on each sample, that is, for each training sample We use the cross entropy loss value calculated in the current training cycle As the weighting coefficient, the following loss-based adaptive regularization term is constructed:

[0095]

[0096] By adopting the dynamic constraint mechanism shown in formula (8), samples with higher loss values ​​(which are more likely to be noise data) will be given a larger weight coefficient. As the training process progresses and the quality of the representation improves, the sample loss value gradually decreases, and the constraint strength of the representation calibration also decreases accordingly, thereby dynamically adjusting the calibration strength for each sample. This dynamic weighting mechanism has a dual adaptive feature, allowing the model to focus on potential noise samples while moderately reducing the constraints on clean samples, thereby helping to learn better representations.

[0097] S3. Stage 3: Combine the cross entropy loss under noise supervision and the representation calibration loss established in the above analysis to train the model.

[0098] S31. Construct the overall objective function as follows:

[0099]

[0100] Where λ1 and λ2 are hyperparameters used to control the strength of representation calibration. λ1 is always kept greater than zero to ensure stable and invariant representation calibration. In complex noise scenarios, λ2 is set to be greater than zero to provide dynamic personalized constraints. The second and third terms of the above loss function are combined to express as:

[0101]

[0102] S32. Perform model training based on the overall objective function.

[0103] The complete algorithm flow of the noise-free prototype-guided characterization calibration method proposed in the present invention is detailed in Algorithm 1.

[0104]

[0105] Finally, it should be pointed out that although the present invention has been described with reference to the current specific embodiments, ordinary technicians in this technical field should realize that the above embodiments are only used to illustrate the present invention and are not used to limit the present invention. Various equivalent changes or substitutions can be made without departing from the concept of the present invention. Therefore, as long as the changes and modifications to the above embodiments are within the scope of the essential spirit of the present invention, they will fall within the scope of the claims of the present invention.

Claims

1. A noiseless prototype-guided characterization calibration method that is resistant to label noise, characterized in that: The following steps are involved: S1. Stage 1: Representation enhancement stage, obtaining enhanced deep representations through unsupervised contrastive learning pre-training; S2. Stage 2: Representation calibration. Using the noise-independent and fixed prototypes obtained in the first stage, we calibrate the biased representations under noise supervision. By imposing directional constraints, we guide the representations affected by noise to the correct position in the feature space. S3. Stage 3: Combine the cross entropy loss under noise supervision and the representation calibration loss established in the above analysis to train the model.

2. The noiseless prototype-guided characterization calibration method resistant to label noise according to claim 1, characterized in that: Said S1 comprises the following steps: S11 adopts the MoCo framework, pre-trains the encoder network, and uses a momentum-based update mechanism to learn visual representations by comparing positive and negative sample pairs in a dynamic memory bank; S12. Construct a contrastive loss using positive and negative sample pairs to optimize the feature extractor.

3. The noiseless prototype-guided characterization calibration method resistant to label noise according to claim 2, characterized in that: The S11 includes the following steps: S111.MoCo framework for each input sample x i , generate its two augmented views: one is the query view x i q , the other is the key view x i k ; S112. Set the query view x i q and key view x i k Input to two encoders with the same network structure but different initializations, and obtain low-dimensional embedding features z through mapping. i q and z i k+ , query view x i q and the corresponding key view x i k constitutes a positive sample pair, and the remaining key views in the current batch are used as negative samples.

4. The noiseless prototype-guided characterization calibration method resistant to label noise according to claim 3, characterized in that: The S12 includes the following steps: S121. For input sample x i , the contrastive loss is defined as: where {k + , k - } represents the set of positive and negative bonds, τ>0 is the temperature parameter; S122. Minimize the loss function shown in the contrast loss formula to train an enhanced contrast representation; S123. Retain the query encoder f′(·) with parameter θ0 for subsequent initial feature extraction.

5. The noiseless prototype-guided characterization calibration method resistant to label noise according to claim 1, characterized in that: The S2 comprises the following steps: S21. Construction and assignment of noise-free prototypes: Assign a prototype to each instance to generate a set of category prototypes. For each instance, assign the category prototype closest to it to form a fixed and noise-independent prototype set corresponding to all samples. S22. Explicitly calibrating biased representations under noisy conditions via consistency regularization constraints and loss-based dynamic regularization constraints.

6. The noiseless prototype-guided characterization and calibration method resistant to label noise according to claim 5, characterized in that: The S21 includes the following steps: S211. Through pre-trained encoder Extract unsupervised representations for all training data: in, Represents sample x i Corresponding unsupervised deep representation; S212. After extracting the unsupervised deep representation, perform k-means clustering on Z and record the prototypes representing each category Where K is the total number of categories, c is the prototype set representing all categories, and based on this prototype set, the Euclidean distance between the unsupervised representation of each sample and all prototypes is calculated as follows: S213. For each instance, assign the category prototype closest to it to form a prototype set corresponding to all samples 7. The noiseless prototype-guided characterization and calibration method resistant to label noise according to claim 5, characterized in that: The S22 includes the following steps: S221. Consistency Regularization Constraint: Under noisy supervision, use a pre-trained feature extractor to initialize the representation and optimize the feature extractor and classifier simultaneously. For classification tasks, add a linear head and use the following cross-entropy loss: in, Noise label One-hot encoding of Providing noisy but valuable supervisory information, together with the prior knowledge provided by the unsupervised prototype, guides representation learning. The following distance constraints guide the subsequent learned representation to be consistent with its assigned prototype C: S222. Loss-based dynamic regularization constraint: Use the size of the loss value to impose dynamic representation calibration constraints on each sample, that is, for each training sample We use the cross entropy loss value calculated in the current training cycle As the weighting coefficient, the following loss-based adaptive regularization term is constructed:

8. The noiseless prototype-guided characterization and calibration method resistant to label noise according to claim 5, characterized in that: The S3 includes the following steps: S31. Construct the overall objective function as follows: Where λ1 and λ2 are hyperparameters used to control the strength of representation calibration. λ1 is always kept greater than zero to ensure stable and invariant representation calibration. In complex noise scenarios, λ2 is set to be greater than zero to provide dynamic personalized constraints. The second and third terms of the above loss function are combined to express as: S32. Perform model training based on the overall objective function.

Images