Progressive Neural Network Training with Selective Weight Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Classical artificial neural networks require significant time and resources for training, are prone to 'network paralysis' and 'freezing at a local minimum,' and have limitations in adapting to changes in image sequencing during training, making them inefficient and resource-intensive.
Innovation Solution
The progressive neural network (p-net) addresses these issues by using corrective weights that are adjusted based on input values and impact coefficients, allowing for rapid training with fewer epochs and enabling continued training without reinitialization, even when image sequences change.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If classical neural networks are trained using traditional methods, then they can learn patterns from data, but training requires significant time and computational resources
Solution Approach 1:
The patent segments the weight adjustment process by introducing distributors that selectively activate specific corrective weights based on input value ranges. Instead of adjusting all weights uniformly, the system divides the weight space into segments controlled by distributors, allowing faster convergence by focusing adjustments on relevant weight subsets.
Solution Approach 2:
The patent implements preliminary action by pre-organizing corrective weights into groups associated with specific input value ranges. The distributors are pre-configured to select appropriate weight groups based on input characteristics, enabling the network to skip unnecessary weight adjustments and converge faster during training.
2Adaptability or versatility
If classical neural networks are trained with fixed weight structures, then training is computationally intensive, but the networks cannot adapt to changes in image sequencing
Solution Approach 1:
The patent introduces dynamic adaptability through distributors that can selectively activate different corrective weight groups based on input characteristics. This dynamic selection mechanism allows the network to adapt to changing data patterns (such as different image sequences) without requiring complete retraining, reducing computational resource usage while maintaining versatility.
Solution Approach 2:
The patent changes the parameter selection mechanism by introducing distributors that modify which weights are active based on input value ranges. This parameter-based selection allows the network to adapt to different input distributions and sequencing patterns by dynamically changing which weight parameters are engaged, rather than using a fixed weight structure.
3Reliability
If classical neural networks use uniform weight adjustment, then training is straightforward, but the networks suffer from 'network paralysis' and 'freezing at a local minimum'
Solution Approach 1:
The patent segments the weight adjustment process into multiple independent corrective weight groups, each controlled by distributors. This segmentation prevents network paralysis by allowing different weight groups to be adjusted independently based on their relevance to current inputs, avoiding the stagnation that occurs with uniform weight adjustment while managing complexity through modular organization.
Solution Approach 2:
The patent applies local quality by allowing different corrective weight groups to be adjusted with different priorities and selection criteria. The distributors enable local optimization by focusing adjustments on weight groups most relevant to current input patterns, improving convergence reliability without requiring complex global adjustments across all weights.
Data Source
AI summary
A neural network includes a plurality of inputs for receiving input signals, and synapses connected to the inputs and having corrective weights. The network additionally includes distributors. Each distributor is connected to one of the inputs for receiving the respective input signal and selects one or more corrective weights in correlation with the input value. The network also includes neurons. Each neuron has an output connected with at least one of the inputs via one synapse and generates a neuron sum by summing corrective weights selected from each synapse connected to the respective neuron. Furthermore, the network includes a weight correction calculator that receives a desired output signal, determines a deviation of the neuron sum from the desired output signal value, and modifies respective corrective weights using the determined deviation. Adding up the modified corrective weights to determine the neuron sum minimizes the subject deviation for training the neural network.