Foundation Model Data Augmentation for AI Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sentence classification techniques require a large amount of labeled data for training machine learning models, which is time-consuming and costly due to the need for manual annotation, and they struggle with efficiently using small amounts of labeled data.
Innovation Solution
The method involves encoding in-distribution and out-of-distribution datasets using a foundation model, pairing components from both datasets in the same or different classes using contrastive learning, and generating an augmented training dataset to improve model training efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large amount of labeled data is used to train machine learning models, then model accuracy is improved, but the time and cost for manual annotation increases
Solution Approach 1:
The patent uses foundation models to generate synthetic training data that copies and adapts from existing in-distribution data. The foundation model learns patterns from a large corpus of in-distribution data and generates out-of-distribution training examples that mimic the target domain's data characteristics, thereby reducing the need for manual annotation of large amounts of labeled data while maintaining model accuracy
Solution Approach 2:
The patent introduces an intermediary foundation model that bridges the gap between in-distribution data and the target out-of-distribution data. The foundation model acts as a mediator that transforms easily obtainable in-distribution data into useful out-of-distribution training data through its encoding and data generation capabilities, eliminating the need for direct manual annotation of the target domain
2Measurement precision
If a large amount of labeled data is used to train machine learning models, then model accuracy is improved, but the cost of data collection and annotation increases
Solution Approach 1:
The foundation model creates synthetic copies of training data by learning from in-distribution data and generating out-of-distribution examples. These synthetic copies replicate the essential characteristics and patterns of the target domain data, providing sufficient training material without requiring collection and labeling of large quantities of real-world data
Solution Approach 2:
The system uses the foundation model's own learned representations and encoding capabilities to generate training data autonomously. The foundation model serves itself by utilizing its pre-trained knowledge from in-distribution data to create out-of-distribution training examples, eliminating the need for external manual annotation resources
3Productivity
If contrastive learning is used to pair in-distribution and out-of-distribution data, then data utilization efficiency is improved, but model complexity increases
Solution Approach 1:
The foundation model serves multiple functions: it encodes in-distribution data, generates out-of-distribution data, and provides representations for contrastive learning. By making the foundation model multi-functional, the patent avoids adding separate specialized components for each task, thereby improving data utilization efficiency without proportionally increasing overall system complexity
Data Source
AI summary
A method, computer system, and a computer program product are provided for data augmentation for training an artificial intelligence (AI) engine. The technique comprises encoding an in-distribution dataset having a plurality of components and an out-of-distribution dataset also having a plurality of components. The encoding is performed using a foundation model. The techniques also comprises pairing one in-distribution component from the dataset with an out-of-distribution component from the dataset in a same class to provide a first set of paired component and pairing another in-distribution component with another out-of-distribution component in a different class using the contrastive learning model to provide a second set of paired components. The first and second set of pared components are then augmented to generate an augmented training dataset. The foundation models is adjusted by using he augmented training dataset to train the AI engine.