Machine Learning Data Augmentation via Affinity Metric Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data augmentation techniques for machine learning models are computationally expensive and inefficient, requiring substantial GPU hours to select effective augmentation methods, which hinders fast and reliable performance improvement.
Innovation Solution
A method and system that iteratively apply default and candidate transformations to training data, compute accuracy metrics, and update transformation lists based on affinity metrics to optimize data augmentation, focusing on individual transformations rather than combinations, thereby reducing computational overhead and improving model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If current data augmentation techniques (AutoAugment, Fast AutoAugment) are used to select effective augmentation methods, then model performance is improved, but computational cost increases substantially (several thousand GPU hours)
Solution Approach 1:
The patent segments the data augmentation process into distinct phases: a fast search phase using RandAugment to identify promising augmentation policies, and a subsequent training phase using those policies. This segmentation allows the system to avoid the computationally expensive exhaustive search of AutoAugment while still finding effective augmentation strategies, thereby resolving the contradiction between model performance and computational cost
Solution Approach 2:
The patent performs preliminary action by conducting a rapid search phase using RandAugment to pre-identify effective data augmentation policies before the main training process. This preliminary selection of augmentation strategies enables the system to achieve good model performance without incurring the substantial computational costs of exhaustive search methods, as the expensive training phase uses already-optimized augmentation policies
2Productivity
If RandAugment technique is used to approximate search space, then search phase speed is improved, but nonnegligible computational cost remains for tuning
Solution Approach 1:
The patent employs cheap short-living objects by using a lightweight, fast search phase with RandAugment that requires minimal computational resources compared to exhaustive search methods. The search phase uses simple random augmentation with fixed parameters to quickly identify promising policies, sacrificing some search thoroughness for speed, thereby resolving the contradiction between search phase speed and tuning computational cost
3Reliability
If data augmentation is applied to increase training dataset size, then overfitting is prevented and model performance is improved, but computational overhead increases
Solution Approach 1:
The patent applies dynamics by using different data augmentation strategies at different stages of the training process. The search phase uses fast, simple RandAugment with fixed parameters, while the training phase uses the identified augmentation policies dynamically. This dynamic approach allows the system to benefit from data augmentation for preventing overfitting while managing computational overhead through efficient phase-specific strategies
Data Source
AI summary
The disclosed systems, structures, and methods are directed to receiving a training data set comprising a plurality of original training samples, augmenting the original training samples by applying default transformations, training the machine learning model on at least a portion of the original training samples and at least a portion of the first set of augmented training samples, computing an unaugmented accuracy, augmenting the original training samples and the first set of augmented training samples by applying a candidate transformation, training the machine learning model on at least a portion of the original training samples, at least a portion of the first set of augmented training samples, and at least a portion of the second set of augmented training samples, computing an augmented accuracy, computing an affinity metric from the unaugmented accuracy and the augmented accuracy, and updating the candidate augmentation transformations list and the default augmentation transformations list.