Neural Network Learning Device Layer Region Sizing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning by neural networks faces significant challenges with overfitting when the amount of learning data is small, leading to reduced generalization performance due to inadequate learning in lower-order layers while high-order layers dominate error reduction through back propagation.
Innovation Solution
A neural network learning device and method that determine the size of local regions for learning in each hierarchical layer based on the network structure, repeatedly extracting and learning from these regions as their size changes, ensuring balanced learning across layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If back propagation is used to learn deep neural networks with limited learning data, then error reduction in high-order layers is achieved, but learning in low-order layers does not advance and overfitting occurs
Solution Approach 1:
The patent applies local quality by determining different local region sizes for different hierarchical layers. Specifically, it sets a first local region size for a first hierarchical layer and a second local region size for a second hierarchical layer, where the ratio of the first local region size to the second local region size is greater than the ratio of the number of neurons in the first hierarchical layer to the number of neurons in the second hierarchical layer. This ensures that lower-order layers receive appropriate learning attention with larger local regions, preventing overfitting while maintaining error reduction capabilities in higher-order layers.
2Productivity
If learning is performed only in high-order layers close to output layer with small learning data, then error is sufficiently decreased, but learning in low-order layer does not advance and neural network gets into overfitted state
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the local region size parameter for different hierarchical layers. The determination means calculates appropriate local region sizes based on the number of neurons in each layer and the desired learning progression. This parameter adjustment ensures that learning data is appropriately distributed across all layers, with lower-order layers receiving larger local regions to prevent overfitting while maintaining overall learning efficiency.
Data Source
AI summary
A neural network learning device 20 is equipped with: a determination module 22 that determines the size of a local region in learning information 200 which is to be learned by a neural network 21 containing multiple layers, said determination being made for each layer, on the basis of the structure of the neural network 21; and a control module 25 that, on the basis of size of the local region as determined by the determination module 22, extracts the local region from the learning information 200, and performs control such that the learning of the learning information represented by the extracted local region by the neural network 200 is carried out repeatedly while changing the size of the extracted local region, and thus, a reduction in the generalization performance of the neural network can be avoided even when there is little learning data.