Neural Network Training with Dynamic Intra-Loop Data Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require large datasets for effective training, which can be time-consuming and expensive to collect, leading to potential bias and overfitting when the available data is limited or biased, resulting in suboptimal performance on diverse datasets.
Innovation Solution
The method involves dynamic data augmentation using affine transformations before and during the training loop, with a data augmentation variable updated based on total loss and class accuracy to generate a more generalized dataset for neural network training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large corpus of training data is collected to meet performance criteria, then neural network performance is improved, but data collection becomes time-consuming and expensive
Solution Approach 1:
The patent applies data augmentation techniques to create synthetic copies of existing training data through transformations such as rotation, flipping, and scaling. This allows the model to train on an expanded dataset without actually collecting more real-world data, thereby improving performance while avoiding the time-consuming data collection process
Solution Approach 2:
The patent performs data augmentation before and during the training process, pre-processing the data to create varied examples in advance. This preliminary action ensures that the model receives diverse training examples without requiring subsequent data collection efforts, resolving the contradiction between performance improvement and time loss
2Loss of time
If a limited or biased training dataset is used, then data collection time is reduced, but the trained neural network becomes biased and subject to overfitting
Solution Approach 1:
The patent generates synthetic data copies through augmentation transformations that diversify the training set. By creating multiple variations of existing data points, the model learns to generalize better without being limited to the original biased dataset, thus avoiding overfitting while maintaining short data collection time
Solution Approach 2:
The patent dynamically adjusts data augmentation parameters during training, such as the degree of rotation, scaling factors, and transformation intensity. This allows the model to adapt to different data characteristics and reduce bias by varying the augmentation parameters, improving performance on diverse data without extending data collection time
3Quantity of substance
If data augmentation is performed to increase dataset size, then the dataset becomes larger and more diverse, but the training process becomes more complex
Solution Approach 1:
The patent implements self-service data augmentation where the training system automatically generates and applies augmentations without external intervention. The augmentation process is integrated into the training loop, with the system dynamically selecting and applying appropriate transformations based on the data characteristics, thereby increasing dataset size while minimizing added complexity
Solution Approach 2:
The patent employs dynamic data augmentation where the augmentation strategy adapts during training based on model performance and data characteristics. The augmentation parameters and strategies are adjusted in real-time, allowing the system to optimize the balance between dataset diversity and training complexity, ensuring efficient learning without excessive computational overhead
Data Source
AI summary
Methods and systems are provided for training a neural network with augmented data. A dataset comprising a plurality of classes is obtained for training a neural network. Prior to initiation of training, the dataset may be augmented by performing affine transformations of the data in the dataset, wherein the amount of augmentation is determined by a data augmentation variable. The neural network is trained with the augmented dataset. A training loss and a difference of class accuracy for each class is determined. The data augmentation variable is updated based on the total loss and class accuracy for each class. The dataset is augmented by performing affine transformations of the data in the dataset according to the updated data augmentation variable, and the neural network is trained with the augmented dataset.