Data Lake Text Generation for ML Training Data Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models for text-based processing require large amounts of high-quality, domain-specific training data, which is difficult to obtain, especially for sensitive domains like medical or financial text, due to data scarcity and quality issues, leading to inefficiencies in model training.
Innovation Solution
A data lake-based approach that utilizes a large repository of textual data to generate new training samples through language processing services, ensuring they are semantically similar yet diverse enough to avoid overfitting, allowing for the creation of high-quality custom ML models with minimal user-provided samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a custom ML model is trained for a particular usage scenario, then model performance is improved, but the amount of training data required increases
Solution Approach 1:
The patent uses pre-trained language models to generate synthetic training data that copies the linguistic patterns and structures from the pre-trained model. This allows creation of domain-specific training samples without manually collecting large amounts of real data, thus maintaining custom model performance while reducing data requirements
Solution Approach 2:
The patent applies transfer learning by pre-training language models on large general-purpose corpora before fine-tuning them for specific domains. This preliminary action on large datasets enables the model to learn general language patterns, reducing the amount of domain-specific data needed for subsequent custom training
2Reliability
If large amounts of high-quality domain-specific training data are used, then model performance is improved, but data acquisition difficulty increases
Solution Approach 1:
Instead of acquiring difficult-to-obtain domain-specific data, the system copies linguistic patterns from readily available pre-trained models to generate synthetic training data. This approach maintains data quality while dramatically easing the acquisition process, especially for sensitive domains like medical or financial text
Solution Approach 2:
The patent uses pre-trained language models as intermediaries that bridge the gap between readily available general-purpose data and required domain-specific training data. The model acts as a mediator that transforms general language knowledge into domain-specific training samples without requiring direct access to sensitive domain data
3Productivity
If training data is augmented to increase quantity, then model training is improved, but risk of overfitting increases
Solution Approach 1:
The patent employs data augmentation techniques that transform existing training samples through various parameter changes such as synonym replacement, random insertion, and back-translation. These transformations create diverse training variations that increase training data quantity while maintaining semantic meaning, thereby improving model training quality without causing overfitting
Data Source
AI summary
Techniques for data lake-based text generation and data augmentation for machine learning training are described. A user-provided dataset including documents and corresponding label information can be automatically supplemented by creating additional high-quality document samples, with labels, via a large repository of documents in a data lake. Documents from the data lake may be identified as being semantically similar to the user-provided documents but different enough to allow a resulting model to learn from the variation in these documents. New documents can be generated from user-provided document samples or data lake sample documents by identifying and replacing slots within the samples and rewriting adjunct tokens.