ML Training Data Conflict Detection and Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning systems face performance degradation due to ground truth conflicts caused by mislabeled or vaguely labeled training data, leading to inconsistencies in class labels, which affect model quality and accuracy.
Innovation Solution
A method is introduced that detects conflicts in training data using conflict detection systems, which employ ground truth clustering with cross-validation and decision space clustering to identify mislabeled data, and presents these conflicts to an oracle for resolution, allowing for adjustment of labels to ensure consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ground truth clustering with cross-validation and decision space clustering are used to detect conflicts in training data, then the quality and accuracy of training data is improved, but the complexity of the training process increases
Solution Approach 1:
The conflict detection system performs ground truth clustering with cross-validation and decision space clustering before the actual training process to identify and resolve labeling conflicts in advance. This preliminary action ensures that only high-quality, conflict-free training data is used for model training, thereby improving labeling accuracy without adding complexity to the core training process.
Solution Approach 2:
The patent introduces an intermediary conflict detection system that acts as a mediator between raw training data and the machine learning model. This intermediary layer analyzes training data for conflicts using clustering techniques and resolves them before data reaches the training process, effectively separating the quality assurance function from the training process itself.
2Stability of the object's composition
If conflicts in training data are detected and resolved using clustering techniques, then the consistency of class labels is improved, but the time required for data preparation increases
Solution Approach 1:
The conflict detection and resolution process is performed as a preliminary step before model training begins. By executing ground truth clustering with cross-validation and decision space clustering in advance, the system identifies and resolves label inconsistencies before they can affect training, ensuring label consistency while allowing the actual training process to proceed efficiently without interruptions.
Data Source
AI summary
A method optimizes a training of a machine learning system. A conflict detection system discovers a conflict between a first training data and a second training data for a machine learning system, where the first training data and the second training data are ground truths that describe a same type of entity, and where the first training data and the second training data have different labels. In response to discovering the conflict between the first training data and the second training data for the machine learning system, an oracle adjusts the different labels of the first training data and the second training data. The machine learning system is then trained using the first training data and the second training data with the adjusted labels.