Data Labeling Priority via Inference Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for data labeling in machine learning model training is hindered by the manual and time-consuming nature of the process, particularly as the amount of data required grows exponentially, necessitating an efficient solution for determining labeling priority.
Innovation Solution
A computer program and computing device method that generates data subsets, trains multiple inference models on these subsets with different validation datasets, and provides labeling priorities based on uncertainty values to optimize the labeling process, allowing for pseudo labeling and prioritization of data that requires manual labeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual data labeling is performed to ensure high quality labels for machine learning model training, then labeling accuracy is improved, but labeling efficiency deteriorates due to the time-consuming nature of manual operations
Solution Approach 1:
The system enables self-service labeling by training inference models to automatically generate pseudo-labels for unlabeled data. The models serve themselves by performing the labeling function that would otherwise require human operators, thereby maintaining accuracy while dramatically improving efficiency.
Solution Approach 2:
Inference models act as intermediaries between the unlabeled data and the final labeled dataset. Instead of direct manual labeling, the system introduces trained models as mediators that process data and generate pseudo-labels, which are then validated and integrated into the training dataset.
2Reliability
If the amount of labeled data is increased to improve model training quality, then machine learning model performance is improved, but the operational burden and time required for labeling increase exponentially
Solution Approach 1:
The system performs preliminary action by pre-training inference models on initially labeled data before they are deployed to generate pseudo-labels for larger datasets. This preliminary training phase enables the models to efficiently process subsequent data without requiring manual labeling, thereby reducing the time burden for expanding labeled datasets.
Solution Approach 2:
The system creates copies of the labeling function through trained inference models. Once a model learns to label data accurately on a small subset, it can generate numerous pseudo-label copies that replicate the quality of manual labeling without the associated time cost, enabling scalable data expansion.
3Measurement precision
If multiple inference models are trained with different validation datasets to improve labeling reliability, then pseudo-label accuracy is improved, but device complexity and computational resources increase
Solution Approach 1:
The system segments the labeling task by dividing the unlabeled dataset into multiple subsets, each processed by different inference models with specialized validation datasets. This segmentation allows parallel processing and reduces the complexity burden on any single model while collectively improving pseudo-label accuracy through diverse perspectives.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
According to an exemplary embodiment of the present disclosure, a computer program stored in a computer readable storage medium is disclosed. The computer program may include instructions for causing one or more processors to perform the following steps, and the steps may include: generating a data subset including at least two or more data from an unlabeled entire dataset; generating two or more training datasets from the data subset, in which the two or more training datasets includes at least one pair of sampled data subset, and in which each of at least one pair of the sampled data subset includes different validation datasets; and training a plurality of inference models, based on the two or more training datasets and corresponding labeling values.