Neural Network Training with Semi-Supervised Noise Label Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine learning models to high accuracy is challenging due to the difficulty in acquiring clean labels, especially in domains like healthcare, where labeling costs are high and noise labels are prevalent in large datasets.
Innovation Solution
The approach involves randomly splitting a training dataset into two labeled datasets and using a semi-supervised learning scheme to train neural networks, where one network corrects and outputs soft labels for the other, and vice versa, to mitigate the impact of noise labels by leveraging unlabeled data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large-scale labeled training data is used to train machine learning models to high accuracy, then model accuracy is improved, but the cost and difficulty of acquiring clean labels increases significantly
Solution Approach 1:
The patent divides the training dataset into multiple subsets and trains different neural networks on different subsets with different labeling strategies. Each neural network is trained on a specific partition of the data, allowing the system to leverage both labeled and unlabeled data in a segmented manner, thereby reducing the dependency on fully labeled data while maintaining model accuracy.
Solution Approach 2:
The patent introduces an intermediary labeling mechanism where neural networks generate pseudo-labels for unlabeled data, which are then used to train other networks. This intermediary pseudo-labeling process allows the system to convert unlabeled data into useful training signals without requiring manual annotation, thus reducing labeling costs while improving model accuracy.
2Quantity of substance
If large quantities of unlabeled data are used, then data quantity is increased, but the presence of noise labels reduces model reliability
Solution Approach 1:
The patent applies different labeling qualities to different parts of the data. Labeled data is used for training networks that require high accuracy, while unlabeled data is used for training networks that can tolerate or benefit from noise labels. This local differentiation of data quality allows the system to maximize the utility of large datasets while maintaining reliability where needed.
Solution Approach 2:
The patent converts the harmful noise labels in unlabeled data into beneficial training signals by using them to train neural networks that are specifically designed to handle noisy data. The noise labels, which would normally degrade model performance, are instead leveraged to improve the network's robustness and its ability to distinguish between true patterns and noise.
3Reliability
If semi-supervised learning is used to train neural networks on both labeled and unlabeled data, then model robustness to noise labels is improved, but training complexity increases
Solution Approach 1:
The patent segments the training process into multiple independent neural network training tasks, each operating on specific data partitions. This segmentation allows the system to manage training complexity by dividing the overall task into smaller, manageable units, each of which can be optimized independently while contributing to the overall robustness of the model.
Data Source
AI summary
An apparatus includes: one or more processors configured to: randomly split a training data set into a first training data set comprising a first label assigned to first data and a second training data set comprising a second label assigned to second data; train a first neural network using a semi-supervised learning scheme based on the first training data set comprising the first label, and an unlabeled second training data set; and train a second neural network using the semi-supervised learning scheme based on the second training data set comprising the second label, and an unlabeled first training data set.