Dataset Augmentation Using Pre-trained Model Feature Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of finding the best data augmentation functions for machine learning models is hindered by a prohibitively large search space, leading to computationally expensive and unfeasible searches, especially when determining hyperparameters and evaluating model quality.
Innovation Solution
A method that uses a pre-trained machine learning model to score candidate augmentation functions without requiring model training, by comparing feature vectors of augmented datasets with an initial training dataset and a reference dataset to determine an optimal distance, thereby selecting the best augmentation functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the search space of all possible augmentation functions is explored to find the best augmentation functions, then the quality of training dataset is improved, but the computational cost becomes prohibitively expensive and technically unfeasible
Solution Approach 1:
A pre-trained machine learning model is used to evaluate augmentation functions before actual training occurs. This preliminary evaluation using feature vectors and optimal distance calculations allows the system to identify promising augmentation functions without performing full model training, significantly reducing computational cost while maintaining quality assessment capability
Solution Approach 2:
Instead of training expensive full ML models to evaluate each augmentation function, the patent uses lightweight feature vector comparisons with a pre-trained model. This disposable, low-cost evaluation method allows extensive search of augmentation functions without prohibitive computational expense
2Measurement precision
If multiple ML models are trained during search to evaluate augmentation quality, then the accuracy of augmentation selection is improved, but the training process becomes computationally very expensive and unfeasible
Solution Approach 1:
A pre-trained ML model is trained in advance before the augmentation search process. This pre-training enables rapid evaluation of multiple augmentation functions during search without requiring repeated full model training, thus maintaining accurate quality assessment while dramatically improving productivity
Solution Approach 2:
Instead of training multiple complete ML models during search, the patent creates simplified copies in the form of feature vector representations using the pre-trained model. These feature vector copies allow rapid comparison and evaluation without the computational burden of full model training
3Productivity
If the search space is reduced by sampling to speed up the process, then the computational cost is reduced, but the method becomes unsuited when the number of hyperparameters and range results in a search space that is too large to be sampled effectively
Solution Approach 1:
The patent replaces the mechanical sampling approach with a mathematical optimization approach using optimal distance calculations in feature vector space. This substitution allows systematic exploration of the search space guided by the pre-trained model's feature representations, maintaining effectiveness even when sampling would fail due to excessive search space size
Data Source
Figure 1
Figure 2
Figure 3
AI summary
According to an aspect of the current invention, a computer-implemented method for generating a training dataset for training a machine learning, ML, model, the method comprising: determining an initial training dataset; determining a plurality of augmentation functions; applying each of the augmentation functions to the initial training dataset to obtain a plurality of augmented training datasets; applying a pre-trained ML model to each of the plurality of augmented training datasets to generate a plurality of feature vectors, and a feature vector of the initial training dataset; comparing the feature vectors of each of the augmented training datasets with feature vectors of the initial training dataset; based on the comparison, selecting at least one of the augmentation function for augmenting the initial training data to generate training data for training the ML model.