Neural Network Learning Device Adversarial Feature Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for learning deep neural networks require a large amount of training data, leading to overfitting when the number of data is small, and existing techniques such as data augmentation and adversarial pattern generation are inefficient in generating data that improve learning performance.
Innovation Solution
A neural network learning device that includes a feature extraction unit, an adversarial feature generation unit, and a network learning unit to process training data and generate adversarial features that help improve learning performance by learning the neural network to approach a desired output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large amount of training data is used for learning deep neural networks, then learning performance is improved, but data acquisition cost and processing time increase
Solution Approach 1:
The patent applies preliminary action by pre-processing training data to extract features and generate adversarial examples before the actual learning process. The feature extraction unit processes training data in advance to create a feature space representation, and the adversarial example generation unit pre-generates adversarial features that will be used during learning. This preliminary preparation reduces the computational burden during the actual learning phase, thereby reducing data processing time while maintaining learning performance.
Solution Approach 2:
The patent extracts essential features from training data using the feature extraction unit, which processes raw training data to obtain compressed feature representations. This extraction process removes redundant information while preserving the essential characteristics needed for learning. By working with extracted features rather than raw data, the system reduces processing time while maintaining the ability to achieve high learning performance.
2Reliability
If data augmentation is used to increase training data, then overfitting is reduced, but data processing complexity increases
Solution Approach 1:
The patent applies parameter changes by transforming the training data in the feature space rather than the raw data space. The adversarial example generation unit modifies feature vectors by adding carefully crafted perturbations that change the parameters of the feature representation. This approach to data augmentation operates on already-processed features, reducing the complexity compared to augmenting raw data while still effectively preventing overfitting by exposing the model to varied feature configurations.
3Reliability
If adversarial pattern generation is used to improve learning, then learning performance is enhanced, but computational resources required increase
Solution Approach 1:
The patent extracts adversarial features from the feature space representation of training data rather than generating adversarial examples from raw data. The adversarial example generation unit operates on compressed feature vectors, which contain essential information in a condensed form. This extraction approach reduces the computational resources required compared to generating adversarial examples from full-resolution raw data, while still achieving enhanced learning performance through adversarial training.
Data Source
AI summary
A large amount of training data is typically required to perform deep network leaning, making it difficult to achieve using a few pieces of data. In order to solve this problem, the neural network device according to the present invention is provided with: a feature extraction unit which extracts features from training data using a learning neural network; an adversarial feature generation unit which generates an adversarial feature from the extracted features using the learning neural network; a pattern recognition unit which calculates a neural network recognition result using the training data and the adversarial feature; and a network learning unit which performs neural network learning so that the recognition result approaches a desired output.