Training Data Supplementation for Consistent AI Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data labeling methods for neural network training suffer from inconsistent quality due to varying user skill levels, leading to inaccurate labels and increased labeling costs and efforts.
Innovation Solution
A method that involves receiving a first user input for a first data set, supplementing a second user input based on the first input to achieve high accuracy in labeling, thereby reducing labeling costs and efforts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple users perform data labeling independently, then labeling throughput increases, but labeling quality consistency deteriorates due to varying skill levels
Solution Approach 1:
The system introduces an intermediary mechanism (supplementation model) that mediates between multiple user labelings and the final training data. This model learns from high-accuracy operators and automatically supplements or corrects labels from other users, maintaining quality consistency while preserving the productivity benefits of multiple labelers
Solution Approach 2:
The system implements feedback loops where user labeling performance is continuously evaluated, and supplementation decisions are made based on learned patterns from high-accuracy operators. This feedback mechanism ensures that labeling quality is maintained across multiple users while preserving throughput
2Measurement precision
If comprehensive data labeling is performed manually, then data accuracy improves, but labeling cost and effort increase
Solution Approach 1:
Instead of requiring complete manual labeling of all data, the system applies partial manual labeling (only from necessary users) and supplements the remainder automatically using the learned supplementation model. This reduces labeling effort while maintaining data accuracy through selective supplementation
Solution Approach 2:
The supplementation model enables the system to self-service by automatically generating or correcting labels based on learned patterns from high-accuracy operators, reducing the need for continuous manual intervention while maintaining labeling quality
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
Disclosed is a method for managing a local model and a global model on an AI platform, the method performed by one or more processors of a computing device according to an exemplary embodiment of the present disclosure. the method may include: obtaining a plurality of sample data; receiving a first user input for a first data set included in the plurality of sample data; receiving a second user input for a second data set excluding the first data set among the plurality of sample data; and obtaining a supplemented second data set by supplementing the second user input based on the first user input.