Uncertainty Mining Net for Noisy Label Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning methods face challenges in training with noisy labeled data, as they require large amounts of labeled data and tend to overfit on limited training data with corrupted labels, and existing semi-supervised learning methods are unable to directly address label noise.
Innovation Solution
The proposed solution involves an Uncertainty Mining Net (UMN) framework that uses an end-to-end deep generative pipeline to estimate label uncertainty, integrating a variational autoencoder for latent feature representation and a conditional VAE for semi-supervised learning, allowing for improved classifier performance by focusing on clean data and reducing the impact of noisy labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning methods are used to train with labeled data, then the model can learn from observed variables, but the model tends to overfit on limited training data with corrupted labels
Solution Approach 1:
The patent applies preliminary action by estimating label corruption probabilities before the main training process. The system pre-processes the labeled data to identify and quantify noisy labels, then uses these estimates to weight samples during training. This preliminary estimation step allows the model to account for label noise before learning begins, preventing overfitting to corrupted labels while still utilizing the labeled data effectively.
2Quantity of substance
If semi-supervised learning methods are used, then the model can utilize unlabeled data, but existing methods are unable to directly address label noise
Solution Approach 1:
The patent introduces an intermediary component - the label corruption probability estimator - that bridges the gap between semi-supervised learning and noisy label handling. This estimator acts as a mediator by providing corruption probability estimates that guide both the supervised and unsupervised learning processes. The intermediary allows the system to simultaneously utilize unlabeled data while accounting for label noise, combining the benefits of semi-supervised learning with robustness to noisy labels.
3Productivity
If traditional machine learning is applied to noisy labeled data, then training can proceed, but the model performance deteriorates due to corrupted labels
Solution Approach 1:
The patent applies parameter changes by modifying the training objective function to incorporate label corruption probability estimates. Instead of treating all labeled samples equally, the system changes the weighting parameter based on estimated corruption probabilities. This parameter adjustment allows training to proceed efficiently without manual data cleaning, while the modified loss function compensates for noisy labels, maintaining classification accuracy despite the presence of corruption.
Data Source
AI summary
Various embodiments are provided for providing machine learning with noisy label data in a computing environment using one or more processors in a computing system. A label corruption probability of noisy labels may be estimated for selected data from a dataset using temporal inconsistency in a machine model prediction during a training operation in a neural network.