Automated Neural Network Construction via Modular Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for automatically generating artificial neural networks are inefficient in constructing powerful networks and often get stuck in secondary minima during training.
Innovation Solution
A method that progressively adds modules and connections to the neural network, selecting the best variations based on performance, using a greedy approach and adjusting learning rates with a trigonometric function to ensure quick convergence and functional equivalence, incorporating normalization and jump connections for efficient generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing automated methods are used to generate neural networks, then automation is achieved, but the construction efficiency is low and networks get stuck in secondary minima
Solution Approach 1:
The patent segments the neural network construction into modular components (normalization modules, jump connections, non-linear function modules) that can be systematically combined. This segmentation enables efficient automated construction while maintaining training reliability through structured module composition.
Solution Approach 2:
The patent applies preliminary action by pre-defining standardized modules and connection types before network construction. These pre-configured modules (normalization layers, skip connections) are prepared in advance and systematically combined, avoiding the need to optimize entire networks from scratch and preventing secondary minima issues.
2Adaptability or versatility
If the network is deepened or widened to maintain function, then network capability is improved, but construction time increases
Solution Approach 1:
The patent changes parameters systematically by adjusting module configurations (normalization types, connection patterns, non-linear functions) rather than randomly searching. This parameter-based approach enables rapid network deepening and widening while maintaining construction efficiency through structured variation.
Solution Approach 2:
The patent implements nesting by embedding standardized modules within the network architecture (normalization modules within layers, jump connections between layers). This nested structure allows the network to be deepened and widened through systematic module composition, achieving improved capability without proportional increases in construction time.
3Productivity
If a greedy approach is used to select extensions, then construction speed increases, but secondary minima may be expected
Solution Approach 1:
The patent implements feedback mechanisms to evaluate module combinations during construction. By systematically assessing the impact of each added module on network performance and using this feedback to guide subsequent additions, the greedy approach maintains high construction speed while avoiding secondary minima through informed selection.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Method for automatically producing an artificial neural network (60) comprising modules and connections linking these modules, wherein successive modules and/or connections are added to a present output network (60), wherein modules and/or connections to be added are selected at random from a prescribable plurality of possible modules and connections to be added, and wherein a plurality of possible developments (61) of the present output network (60) are each produced by adding modules and/or connections to be added to the output network, and wherein one of the developments (61) from the plurality of possible developments (61) is then selected in order to serve as a present output network (60) in a next pass of the method.