Iterative Data Labeling with Confidence-Based Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In machine learning, especially in supervised learning scenarios, accurately labeling data samples without determined labels is inefficient and costly, particularly in big data applications where manual correction is impractical.
Innovation Solution
A method and apparatus that iteratively train a prediction model using a combination of data samples with and without determined labels, generating additional samples if necessary through Generative Adversarial Networks (GANs), and checking the accuracy of learned labels to refine and correct them until meeting preset requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual correction of labels is performed for all data samples, then labeling accuracy is improved, but time consumption and cost increase significantly
Solution Approach 1:
The patent applies partial action by performing manual verification only on a subset of data samples (those with lower confidence scores or higher importance) rather than all samples. The system automatically labels most samples using the prediction model, then selectively verifies only the necessary portion manually, achieving high accuracy without proportional time investment.
Solution Approach 2:
The system implements self-service through automated prediction models that generate labels independently. The prediction model processes data samples and generates preliminary labels automatically, reducing reliance on manual intervention. The system serves itself by using its own predictions to guide the selective manual verification process.
2Reliability
If all data samples are manually labeled, then labeling completeness is improved, but productivity decreases
Solution Approach 1:
The system performs preliminary automated labeling for all data samples before manual verification. The prediction model generates initial labels for the entire dataset, ensuring completeness is established upfront. This preliminary action allows the system to maintain complete labeling coverage while minimizing subsequent manual work to only verification tasks.
Solution Approach 2:
The prediction model serves as an intermediary between raw data and final labels. It processes all data samples to generate preliminary labels, acting as a bridge that handles the bulk of labeling work. This intermediary then guides the manual verification process by identifying which samples need human review, maintaining completeness while improving efficiency.
3Measurement precision
If a prediction model is trained with more data samples, then model accuracy is improved, but training time and computational resources increase
Solution Approach 1:
The system implements continuous model improvement through iterative training. Instead of single batch training, the model is continuously refined using newly verified labeled data from each iteration. This continuous action allows the model to progressively improve accuracy incrementally, avoiding the need for extensive retraining from scratch and reducing overall training time.
Solution Approach 2:
The training process is made dynamic through iterative refinement. The model adapts its training data composition across iterations, dynamically incorporating newly verified samples. This dynamic approach allows the model to improve accuracy progressively with smaller, manageable training batches rather than requiring all data to be processed simultaneously, reducing computational burden.
Data Source
AI summary
Disclosed are a data sample label processing method and apparatus. The data sample label processing method comprises: obtaining a first set of data samples without determined labels and a second set of data samples with determined labels; performing an iteration with the following steps until an accuracy rate meets a preset requirement: training a prediction model based on a combination of the first set of data samples and the second set of data samples; inputting data samples from the first set of data samples into the prediction model to obtain prediction values as learning labels for each data sample, and associating the learning labels with the data samples respectively; obtaining a subset from the first set of data samples, wherein the subset comprise data samples associated with learning labels; obtaining determined labels for the data samples in the subset; obtaining the accuracy rate based at least on the learning labels of the data samples in the subset and the determined labels of the data samples in the subset; and if the accuracy rate does not meet the preset requirement, labeling the data samples in the subset with the determined labels for the data samples in the subset, and moving the subset from the first set of data samples to the second set of data samples; and after the iteration ends, labeling the remaining data samples in the first set with the associated learning labels.