Label Error Detection via Clustering Training Losses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Supervised machine learning models face inefficacy due to corrupted labels in datasets, leading to poor generalization, and existing methods are inadequate for detecting label errors effectively.
Innovation Solution
A method and system for detecting label errors using a neural network that trains on noisy datasets, generates training loss data, constructs a loss matrix, applies clustering algorithms to separate clean and noisy labels, and refines the network to remove or replace noisy labels, enhancing classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If machine learning models are trained on noisy datasets, then training data availability is improved, but model accuracy deteriorates
Solution Approach 1:
The patent extracts and removes noisy labels from the training dataset by detecting label errors through clustering training losses. The system identifies and separates clean labels from noisy labels, creating a cleaned dataset that maintains data availability while improving accuracy by excluding corrupted samples from training.
Solution Approach 2:
The patent implements a feedback mechanism where the model's training losses are monitored and used to detect label errors. The clustering algorithm analyzes loss patterns to identify noisy labels, which are then removed or corrected, and the process repeats until convergence. This iterative feedback loop continuously improves data quality while maintaining training data availability.
2Measurement precision
If existing label error detection methods are used, then detection capability is improved, but computational complexity increases
Solution Approach 1:
The patent employs a self-service approach where the model detects its own label errors through self-monitoring of training losses. The clustering algorithm automatically identifies noisy labels based on loss patterns without requiring external intervention or complex additional systems. The model uses its own training signals to diagnose and correct label errors, reducing computational overhead compared to external detection systems.
Data Source
AI summary
Systems and methods for tackling a significant problem in data analytics: inaccurate dataset labeling. Such inaccuracies can compromise machine learning model performance. To counter this, label error detection algorithm is provided that efficiently identifies and removes samples with corrupted labels. The provided framework (CTRL) detects label errors in two steps based on the observation that models learn clean and noisy labels in different ways. First, one trains a neural network using the noisy training dataset and obtains the loss curve for each sample. Then, one applies clustering algorithms to the training losses to group samples into two categories: cleanly-labeled and noisily-labeled. After label error detection, one removes samples with noisy labels and retrains the model.