Active Multifidelity Learning for Language Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The training and utilization of machine learning models are resource-intensive, time-consuming, and costly, often requiring large amounts of computing resources and lacking domain-specific training, which can lead to inefficiencies and inaccuracies, especially when relying on pre-trained models that are not specific to a particular domain.
Innovation Solution
The implementation of an active multifidelity learning process that combines high-fidelity manual labeling with low-fidelity automated labeling by a pre-trained machine learning model, using in-context learning to improve automated labels and iteratively fine-tune a target machine learning model for specific domains, reducing the need for extensive manual labeling and computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling is used for all training data, then labeling accuracy is improved, but time consumption and cost increase significantly
Solution Approach 1:
The patent segments the training data into different subsets that are processed through different labeling pathways. Some data is manually labeled while other data is automatically labeled, allowing the system to balance accuracy and efficiency by treating different portions of data differently based on their specific requirements.
Solution Approach 2:
The patent introduces an intermediary verification mechanism where automatically labeled data is reviewed and validated by manual labelers or through additional automated checks. This intermediary step ensures that automated labeling maintains sufficient accuracy while reducing the overall time consumption compared to purely manual labeling.
2Productivity
If pre-trained machine learning models are used, then training speed is improved, but domain-specific accuracy deteriorates
Solution Approach 1:
The patent performs preliminary domain-specific data preparation and curation before training the model. By pre-processing and selecting domain-relevant data in advance, the system maintains training speed while ensuring the model learns domain-specific patterns effectively.
Solution Approach 2:
The patent applies different training strategies to different components or layers of the model. Certain parts of the model are fine-tuned with domain-specific data while other parts retain their general capabilities, allowing the system to maintain both speed and domain-specific accuracy through differentiated treatment of model components.
3Reliability
If extensive manual labeling is performed, then training data quality is improved, but computing resource utilization increases
Solution Approach 1:
The patent applies partial manual labeling only to the most critical or ambiguous portions of the training data rather than labeling all data manually. This selective approach maintains sufficient data quality for reliable training while significantly reducing the computing resources required compared to comprehensive manual labeling of entire datasets.
4Adaptability or versatility
If domain-specific training data is collected manually, then model specialization is improved, but productivity decreases
Solution Approach 1:
The patent implements a self-service automated data collection and labeling system that uses pre-trained models to generate domain-specific training data automatically. This self-service approach eliminates the need for manual data collection while still producing specialized training data, thereby maintaining model specialization capabilities while dramatically improving productivity.
Data Source
AI summary
Aspects of the present disclosure provide techniques for active multifidelity machine learning. Embodiments include selecting, based on one or more criteria, a first subset of unlabeled training data for manual review and a second subset of unlabeled training data for providing to a pre-trained machine learning model for automated labeling. Embodiments include receiving manual label data for the first subset of unlabeled training data. Embodiments include providing inputs to the pre-trained machine learning model based on a subset of the manual label data and the second subset of training data. Embodiments include receiving, as outputs from the pre-trained machine learning model, automated label data for the second subset of unlabeled training data. Embodiments include generating a training data set for a target machine learning model based on the set of unlabeled training data, the manual label data, and the automated label data.