Few-Shot Learners for Automated Data Label Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in machine learning is the time-consuming and error-prone process of manually verifying labels in datasets, especially as datasets grow, making it impractical for large-scale applications, and existing methods rely heavily on high-quality labeled data, which is not always available.
Innovation Solution
A method involving dividing datasets into training and inference sets, training label-specific few-shot learners, and using them to generate predicted label output values, which are then aggregated to determine dataset verification based on variability thresholds, thereby automating the detection of mislabeled data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual verification of labels is performed, then label quality is improved, but time consumption increases significantly
Solution Approach 1:
The system uses the dataset itself to verify its labels by training few-shot learners on the data and using these learners to predict and verify labels automatically, eliminating the need for external manual verification while maintaining high label quality
Solution Approach 2:
Few-shot learners act as intermediaries between the raw dataset and the verification process, enabling automated label verification without direct human intervention while preserving label quality through learned patterns
2Loss of time
If automated detection of mislabeled data is implemented, then time consumption is reduced, but measurement precision deteriorates
Solution Approach 1:
The dataset is divided into training and inference subsets, and multiple few-shot learners are trained on different training subsets. This segmentation allows the system to process large datasets efficiently while maintaining detection accuracy through ensemble predictions from multiple specialized learners
Solution Approach 2:
The system changes the parameter of training data quantity by using few-shot learning (small number of training examples) to enable automated verification with limited computational resources while maintaining detection precision through the aggregation of predictions from multiple learners
3Reliability
If high-quality labeled data is used for training, then model performance is improved, but device complexity increases
Solution Approach 1:
The system generates its own training data by randomly sampling from the dataset and using these samples to train few-shot learners, eliminating the need for external high-quality labeled datasets while maintaining model performance through self-generated training examples
Solution Approach 2:
The system dynamically creates training datasets by random sampling during the verification process, allowing the training data composition to change across different verification runs. This dynamic approach enables the system to adapt to different data distributions and maintain performance without requiring fixed high-quality labeled training sets
Data Source
AI summary
Aspects of the present invention disclose a method for verifying labels of records of a dataset. The records comprise sample data and a related label out of a plurality of labels. The method includes one or more processors dividing the dataset into a training dataset comprising records relating to a selected label and an inference dataset comprising records with sample data relating to the selected label and all other labels out of the plurality of labels. The method further includes dividing the training dataset into a plurality of learner training datasets that comprise at least one sample relating to the selected label. The method further includes training a plurality of label-specific few-shot learners with one of the learner training datasets. The method further includes performing inference by the plurality of trained label-specific few-shot learners on the inference dataset to generate a plurality of sets of predicted label output values.