Deep Learning Label Correction for Noisy Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning models are prone to performance deterioration due to the use of noisy labels, especially in high-cost labeling scenarios like medical systems and manufacturing lines, where wrong labels are inevitable, necessitating a robust model against such errors.
Innovation Solution
A machine learning apparatus with a label correction unit that iteratively updates initial labels based on model outputs, a learning rate estimation unit to determine an optimal learning rate, and a label-noise-resistant training unit to ensure reliable target labels, using a balance between suppressing overfitting and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If supervised learning with manually added target labels is used to improve model reliability, then model performance is improved, but wrong labels are inevitably contained causing significant deterioration in model performance
Solution Approach 1:
The system enables the model to self-correct labels by using its own predictions to identify and correct wrong labels in the training data. The model iteratively improves its own training data quality without requiring external expert intervention for each label correction.
Solution Approach 2:
The system implements a feedback mechanism where the model's predictions are fed back into the training process to identify and correct wrong labels. The correction unit uses the model outputs to determine which labels are incorrect and updates them accordingly, creating a closed-loop system that continuously improves data quality.
2Manufacturing precision
If the number of label updates is increased to improve label accuracy, then more wrong labels are corrected, but training time and computational cost increase
Solution Approach 1:
The system performs partial updates by selectively correcting only the labels that are identified as wrong based on model predictions, rather than updating all labels. This approach corrects sufficient labels to achieve high accuracy without the computational cost of exhaustive updates.
Solution Approach 2:
The system performs preliminary identification of wrong labels using model predictions before conducting the actual label correction. This preliminary action allows the system to target only the necessary labels for correction, avoiding unnecessary computational work on already correct labels.
3Productivity
If a high learning rate is used to speed up training convergence, then training speed is improved, but the model may overfit to wrong labels
Solution Approach 1:
The system dynamically adjusts the learning rate based on the correction status of labels. The learning rate is set to be higher when wrong labels are present to accelerate convergence, and adjusted as corrections are made, allowing the model to adapt its training behavior to the current data quality state.
Solution Approach 2:
The system changes the learning rate parameter in response to label correction progress. By modifying this critical training parameter based on the identified wrong labels and correction status, the system optimizes both training speed and robustness throughout the training process.
Data Source
AI summary
According to one embodiment, a machine learning apparatus comprising a processor. The processor acquires, by applying a first sample to a first deep learning model that processes a classification problem, a first model output containing an inference probability and/or a feature vector output. The processor determines whether an update of a label to be used as a teacher in learning of the first deep learning model is required, based on the first model output and/or the label. The processor updates the label based on the first model output and a label at a current number of updates if it is determined that the update of the label is required, and terminates the update of the label if it is determined that the update of the label is not required.