Neural Network Learning Device Adaptive Noise Injection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for suppressing overfitting in neural networks, such as optimizing node values and masking input data, face challenges in setting parameters like noise frequency, position, and size, leading to insufficient suppression of overfitting.
Innovation Solution
A learning device that adds noise to input data and intermediate layer data based on the contribution of elements to the inference result, with noise parameters adaptively determined by the learning progress and recognition rate, using a contribution-degree image to adjust the noise and update neural network parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If noise is added to input data to suppress overfitting, then generalization performance is improved, but parameter setting becomes difficult and overfitting cannot be sufficiently suppressed
Solution Approach 1:
The system automatically determines noise parameters based on contribution degree calculations, eliminating the need for manual parameter setting. The noise adding unit adaptively adjusts noise frequency, position, and size based on how much each element contributes to the inference result, allowing the system to self-optimize without external intervention.
Solution Approach 2:
The patent dynamically changes noise parameters (frequency, position, size) based on contribution degree values. Instead of using fixed noise parameters, the system adjusts them according to the importance of different input elements, transforming static parameter settings into dynamic, adaptive parameter control that responds to the actual data characteristics.
2Ease of manufacture
If manual parameter setting for noise adding is used, then implementation is simple, but overfitting suppression is insufficient
Solution Approach 1:
The system performs preliminary calculation of contribution degrees before adding noise. By pre-calculating which elements contribute most to the inference result, the system can then strategically add noise to those high-contribution elements, ensuring maximum effectiveness in suppressing overfitting while maintaining implementation feasibility through automated processes.
Solution Approach 2:
The system uses contribution degree calculations as feedback to guide noise adding. The noise adding unit continuously adjusts noise parameters based on feedback from contribution degree analysis, creating a closed-loop system that automatically optimizes overfitting suppression without requiring manual parameter tuning or complex external control.
Data Source
AI summary
A learning device includes one or more processors. The processors acquire input data and a target label indicating a correct answer of inference based on the input data. The processors add noise to at least one of the input data and intermediate layer data of the neural network and perform inference by the neural network with respect to the input data. The noise is based on contributions of a plurality of elements included in the input data with respect to an inference result when the input data is input to a neural network. The processors update parameters of the neural network so that the inference result by the neural network matches the target label.