Data Source Evaluation Platform for Supervised Learning Model Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Supervised machine learning models are prone to inaccuracies due to labeling errors in the training data, leading to operational inefficiencies.
Innovation Solution
A computing platform applies unsupervised learning algorithms to identify discrepancies between clustering information and labeling information, flags data labeling errors, and trains supervised learning models using corrected data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If supervised machine learning models are trained using labelled data, then the models can make automated predictions, but labeling errors in the training data reduce reliability and result in inaccurate outputs
Solution Approach 1:
The system performs preliminary clustering analysis on the labelled data set before training the supervised learning model. By applying unsupervised clustering algorithms to identify natural groupings in the data and comparing these with the provided labels, the system detects potential labeling errors in advance, allowing corrections to be made before model training begins.
Solution Approach 2:
The system introduces clustering information as an intermediary mechanism to verify labeling accuracy. The clustering results serve as a reference standard to compare against the provided labels, enabling the detection and correction of labeling errors without requiring manual review of each data point.
2Measurement precision
If unsupervised learning algorithms are applied to identify labeling errors, then data accuracy improves, but the computational complexity and processing time increase
Solution Approach 1:
The system applies clustering algorithms selectively to portions of the data set or uses pre-computed clustering results when available, rather than performing complete re-clustering. This partial action approach maintains sufficient accuracy for error detection while reducing computational overhead compared to exhaustive analysis of all data points.
Solution Approach 2:
The system uses clustering information from the labelled data set as a template or reference to evaluate and correct labels in the unlabeled data set. By copying and adapting the clustering structure, the system extends labeling verification to additional data without incurring the full computational cost of independent clustering analysis.
3Reliability
If data points with discrepancies are removed or corrected, then model training accuracy improves, but the data set size and available training examples decrease
Solution Approach 1:
The system converts the potential harm of reduced training data volume into a benefit by strategically removing only the erroneous data points while retaining the majority of valid training examples. The clustering-based verification ensures that high-quality data points are preserved, and the loss of a small number of incorrect labels results in net improvement of model accuracy.
Solution Approach 2:
The system adjusts the threshold parameters for discrepancy detection and correction, allowing flexible control over the balance between maintaining data volume and ensuring data quality. By tuning these parameters, the system can optimize the trade-off between removing erroneous labels and preserving sufficient training data for effective model training.
Data Source
AI summary
Aspects of the disclosure relate to evaluating sources of training data for model generation. A computing platform may receive, from one or more data sources, a labelled data set. The computing platform may apply, to the labelled data set, an unsupervised learning algorithm, resulting in a clustered data set. The computing platform may compare, for each data point in the labelled data set, corresponding clustering information and labelling information to identify discrepancies. The computing platform may flag, for data points with identified discrepancies between the clustering information and labelling information, a labelling error. The computing platform may grade, based on the flagged labelling errors, each of the one or more data sources. Using remaining data of the labelled data set, not flagged with labelling errors, the computing platform may train a supervised learning model by weighting the remaining data based on: a corresponding data source and its grade.