Data Augmentation Hyperparameter Tuning for DNN Decision Planes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data augmentation methods struggle to produce high-precision AI models due to improper conversion processing, leading to deviations in decision planes from ideal boundaries, which can result in reduced predictability and failure to improve performance.
Innovation Solution
A data augmentation method involving alternately performing first and second optimization processing, where the first optimizes neural network parameters and the second optimizes hyperparameters for data conversion, using partial differentiation to minimize distribution differences between test and augmented training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data augmentation processing is performed using conventional methods, then the number of training samples is increased, but the decision plane deviates from the ideal boundary resulting in reduced predictability
Solution Approach 1:
The patent changes the parameters of data augmentation processing by optimizing hyperparameters (such as rotation angles, scaling factors, and translation distances) to find the optimal transformation parameters that maintain decision plane accuracy while increasing training sample diversity. This resolves the contradiction by adjusting the quality and magnitude of augmentation transformations.
Solution Approach 2:
The patent implements a feedback mechanism where the decision plane accuracy is continuously evaluated during the optimization process, and this feedback is used to adjust the hyperparameters for subsequent data augmentation. The system monitors whether augmented samples are causing decision plane deviation and modifies the augmentation strategy accordingly, ensuring that predictability is maintained while still increasing sample quantity.
2Ease of manufacture
If hyperparameters for data conversion are not optimized, then data augmentation can be performed simply, but the AI model fails to achieve high precision
Solution Approach 1:
The patent enables the system to automatically optimize its own hyperparameters through self-service mechanisms. The optimization process is performed autonomously by the system itself using the training data and evaluation metrics, without requiring manual intervention or external expertise. This resolves the contradiction by making the complex optimization process self-executing while achieving high model precision.
Solution Approach 2:
The patent performs preliminary optimization of hyperparameters before the main training process begins. By pre-optimizing the data conversion parameters using a subset of data or through preliminary experiments, the system prepares the optimal augmentation strategy in advance, which then guides the subsequent training process to achieve high precision without adding complexity during the main training phase.
3Productivity
If conversion processing is performed improperly on training samples, then data augmentation can be executed quickly, but predictability is lowered
Solution Approach 1:
The patent applies partial optimization by focusing computational resources on optimizing only the most critical hyperparameters that have the greatest impact on predictability, rather than exhaustively optimizing all parameters. This selective approach maintains high productivity while ensuring that the key factors affecting reliability are properly addressed. The system identifies and optimizes only the essential conversion parameters.
Data Source
AI summary
First optimization processing for optimizing parameters of a DNN and second optimization processing for optimizing hyperpararneters for each sample used in data augmentation processing are alternately performed. The first optimization processing includes causing the DNN to predict a first augmentation label from a first augmented sample, calculating a first error function between the first augmentation label and a first correct label for a first sample, and updating the parameters in accordance with the first error function. The second optimization processing includes acquiring a second sample, causing the DNN after the updating of the parameters to predict a second label from the second sample, calculating a second error function between the second label and a second correct label for the second sample, and updating the hyperparameter in accordance with a gradient obtained by differentiation of the second error function with respect to the hyperparameter.