Machine Learning Data Augmentation for Classifier Generalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning technologies face challenges in optimizing more complex evaluation metrics, such as constrained evaluation metrics, during the training of classifiers, especially when data augmentation using Mixup is employed, which can lead to biased predictions and suboptimal generalization performance.
Innovation Solution
A machine learning method that estimates the improvement in evaluation metrics by selecting pairs of training data, generating new training data through Mixup, and training the classifier using both the new and original training data, thereby optimizing complex evaluation metrics while improving generalization performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data augmentation using Mixup is employed to improve generalization performance, then generalization performance is improved, but evaluation metrics such as worst recall and average recall under coverage constraints cannot be optimized
Solution Approach 1:
The patent pre-calculates importance scores for training data pairs before performing Mixup augmentation. By estimating the degree of improvement in evaluation metrics (such as worst recall and average recall under coverage constraints) in advance, the system identifies which data pairs should be augmented. This preliminary assessment allows the system to optimize complex evaluation metrics while maintaining generalization performance improvements from Mixup.
Solution Approach 2:
The patent changes the parameter selection criteria for Mixup by introducing importance scores based on estimated evaluation metric improvement. Instead of uniformly applying Mixup to all data pairs or using simple heuristics, the system dynamically adjusts which data pairs receive augmentation based on their potential contribution to optimizing complex evaluation metrics. This parameter-based selection resolves the contradiction by making augmentation targeted rather than random.
2Reliability
If complex evaluation metrics are optimized during training, then prediction bias is reduced, but training complexity increases
Solution Approach 1:
The patent segments the training process into distinct phases: (1) calculating importance scores for data pairs based on complex evaluation metrics, (2) selecting top-ranked pairs for Mixup augmentation, and (3) performing standard training on augmented data. This segmentation allows the system to handle complexity in a structured way, using pre-computed importance scores to guide augmentation without requiring continuous complex metric optimization during the main training loop.
Solution Approach 2:
The patent introduces importance scores as an intermediary metric that bridges complex evaluation metrics and practical training decisions. Instead of directly optimizing complex metrics like worst recall under coverage constraints during training, the system uses pre-calculated importance scores derived from these metrics to guide data pair selection for augmentation. This intermediary simplifies the training process while still achieving optimization of complex evaluation metrics.
Data Source
AI summary
A machine learning method includes estimating a degree of improvement of an evaluation metric related to a machine learning model, the improvement being to be obtained on a condition that the machine learning model is configured to be trained based on combination of training data in a training data set, selecting a pair of training data from the training data set based on the estimated degree, generating other training data based on the selected pair of training data, and training the machine learning model based on the other training data and the training data set, using a processor.