Automatic Label Correction for Deep Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models trained with incorrect labels can misrecognize these labels as correct, leading to degraded performance in accuracy, and existing methods lack effective solutions for automatically correcting such errors.
Innovation Solution
A processor-implemented method involving two models: a first model to predict confidences of labels and a second model to estimate correct labels, using iterative training based on confidence probabilities and regularization penalties, with the EM algorithm, to detect and correct incorrect labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep learning models are trained with incorrect labels, then training can proceed with available data, but the model accuracy degrades due to misrecognizing incorrect labels as correct
Solution Approach 1:
The patent applies preliminary action by detecting and correcting incorrect labels in the training dataset before the deep learning model is trained. The label correction module identifies erroneous labels using confidence scores and alternative label generation, then corrects them beforehand. This ensures the model trains on clean data, maintaining both training efficiency and model accuracy without encountering degraded performance during the training process.
2Reliability
If manual verification of labels is performed to ensure accuracy, then model accuracy improves, but the time and resources required increase significantly
Solution Approach 1:
The patent implements self-service by enabling the system to automatically detect and correct its own labeling errors without human intervention. The label correction module uses the model's confidence scores, generates alternative labels, and automatically corrects incorrect labels. This self-correcting mechanism achieves high label accuracy while eliminating the time and resource costs associated with manual verification.
Solution Approach 2:
The system employs feedback by using the model's confidence scores as indicators of potential labeling errors. When confidence is low or alternative labels are generated, the system feeds this information back to the label correction module, which then corrects the labels. This closed-loop feedback mechanism ensures high accuracy without requiring manual verification.
3Reliability
If all labels are manually verified to ensure correctness, then accuracy improves, but productivity and scalability are reduced
Solution Approach 1:
The patent applies local quality by selectively correcting only the labels that are likely to be incorrect rather than verifying all labels. The label correction module uses confidence scores to identify suspicious labels and applies correction only to those specific cases. This targeted approach maintains high label correctness while preserving overall data processing throughput and scalability.
Solution Approach 2:
The system performs partial action by applying label correction only to a subset of labels that exhibit characteristics of being incorrect (low confidence scores, generated alternative labels). This partial correction approach is sufficient to maintain high accuracy while avoiding the productivity loss that would result from verifying or correcting all labels.
4Ease of manufacture
If no label correction is performed and training proceeds with noisy labels, then the process is simple and fast, but the model robustness deteriorates
Solution Approach 1:
The patent extracts and removes the harmful element (incorrect labels) from the training dataset before training begins. The label correction module identifies and corrects erroneous labels, effectively taking out the noise that would otherwise degrade model robustness. This maintains training process simplicity while significantly improving model reliability and robustness.
Data Source
AI summary
A processor-implemented method includes training a first model to predict confidences of labels for data samples in a training dataset, including using a corrected data sample obtained by correcting an incorrect label based on a corresponding confidence detected by the first model and an estimated corrected label generated by a second model; training the second model to estimate correct labels for the data samples, including estimating a correct other label corresponding to another incorrect label detected based on a corresponding confidence generated by the first model with respect to the other incorrect label; and automatically correcting the other incorrect label with the estimated correct other label.