Neural Network Training via Input Space Mapping Against Overfitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) suffer from overfitting, leading to poor performance when processing new data despite perfect prediction of training sets.
Innovation Solution
A neural network training method that involves determining a first input space based on a first output space to reduce the capability of fitting noise, thereby avoiding overfitting and improving generalization capability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the neural network is trained to perfectly predict the training set, then the training accuracy is improved, but the ability to handle new data deteriorates due to overfitting
Solution Approach 1:
The patent extracts and removes the noise-fitting capability from the neural network by introducing a noise injection mechanism that actively eliminates spurious patterns. The system separates signal from noise by training the network to ignore noisy patterns, thereby removing the harmful overfitting tendency while preserving genuine learning ability.
Solution Approach 2:
The patent converts the harmful noise-fitting capability into a beneficial mechanism by using noise injection during training. The noise that would normally cause overfitting is intentionally introduced and then processed by the network, transforming it into a tool for learning robust patterns. This allows the network to distinguish between meaningful signals and spurious noise, improving generalization.
2Manufacturing precision
If the neural network fits noise patterns, then the training performance is improved, but the performance on new data deteriorates
Solution Approach 1:
The patent applies preliminary noise injection during the training process before the network sees test data. By pre-exposing the network to controlled noise patterns during training, the system prepares the network to recognize and ignore noise in future scenarios. This preliminary action prevents the network from developing spurious correlations that would harm performance on new data.
Solution Approach 2:
The patent changes the parameter of noise level during training by dynamically adjusting noise injection intensity. The system modifies training parameters to include controlled noise, transforming the training process into one that produces robust models. This parameter change allows the network to learn invariant features that generalize better while maintaining high training performance.
Data Source
AI summary
A neural network training method in an artificial intelligence (AI) field includes inputting training data into a neural network; determining a first input space of a second target layer in the neural network based on a first output space of a first target layer in the neural network; and inputting a feature vector in the first input space into the second target layer, where a capability of fitting random noise by the neural network when the feature vector in the first input space is input into the second target layer is lower than a capability of fitting the random noise by using an output space that is in the neural network and that exists when a feature vector in the first output space is input into the second target layer.