Dual Loop Meta-Learning for Noisy Label Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning approaches face challenges in achieving reliable and robust models due to the scarcity of reliable training data, especially for deep neural networks, where inconsistent and noisy labels from various data sources hinder effective learning.
Innovation Solution
The proposed solution involves a dual loop learning process that includes a first loop for data decoding learning and a second loop for label decoding learning. This process updates parameters associated with decoding and label decoding using ground truth labels, allowing for the correction of inconsistent and noisy labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple data sources with inconsistent labels are used for training, then the quantity of training data increases, but the reliability of the training data decreases
Solution Approach 1:
The patent segments the training data into multiple data sources, each with its own label distribution characteristics. By treating each data source separately and then combining them through the joint training framework, the system can leverage the quantity advantage of multiple sources while managing their individual reliability issues through source-specific parameter adaptation.
Solution Approach 2:
The patent introduces source-specific parameters that allow the model to adapt to different label distributions from various data sources. By changing parameters to account for label inconsistency across sources, the system can effectively utilize large quantities of training data from multiple sources without being unduly affected by their individual reliability limitations.
2Reliability
If manual annotation is used to ensure label accuracy, then the reliability of labels improves, but the productivity of data labeling decreases
Solution Approach 1:
The patent enables the system to self-correct label errors by using the joint training framework that leverages consistent patterns across multiple data sources. The model automatically identifies and corrects labeling inconsistencies without requiring manual verification, thus maintaining high label reliability while achieving high productivity through automated processing.
Solution Approach 2:
The patent implements a feedback mechanism where the model's predictions from multiple data sources are continuously compared and used to refine label assignments. This feedback loop allows the system to automatically improve label accuracy through iterative training, eliminating the need for slow manual annotation while maintaining high reliability.
3Device complexity
If deep neural networks are trained with noisy labels, then the model complexity increases, but the manufacturing precision of the model decreases
Solution Approach 1:
The patent modifies the training parameters by introducing source-specific adaptation parameters and joint loss functions that account for label noise. These parameter changes enable the complex deep neural network to robustly handle noisy labels from multiple sources, maintaining high model accuracy despite the increased model complexity required to process diverse data sources.
Solution Approach 2:
The patent creates a composite training framework that combines multiple data sources with different noise characteristics. By treating the training system as a composite structure where each data source contributes differently weighted information, the model can leverage its complexity to handle the heterogeneity of noisy labels while achieving high precision through the synergistic combination of multiple sources.
Data Source
AI summary
The present teaching relates to method, system, medium, and implementations for machine learning. Machine learning is performed based on training data via a dual loop learning process that includes a first loop for data decoding learning and a second loop for label decoding learning. In the first loop, first parameters associated with decoding are updated to generate updated first parameters based on a first label, estimated via the decoding using the first parameters, and a second label, predicted via the label decoding using second parameters. In the second loop, the second parameters associated with the label decoding are updated to generate updated second parameters based on a third label, obtained via the decoding using the updated first parameters, and a ground truth label.