Progressive Neural Network Weight Correction for Training Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Classical neural networks require significant time and resources for training, are limited in expanding their size without full retraining, and are prone to issues like 'network paralysis' and 'freezing at a local minimum', making them inefficient for complex image recognition tasks.
Innovation Solution
The progressive neural network (p-net) addresses these issues by using a weight correction calculator to modify corrective weights based on input values, allowing for rapid training with fewer epochs and the ability to add or remove neural network elements, thereby optimizing training time and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If classical neural networks are trained using traditional methods, then the network can learn patterns and perform recognition tasks, but the training process requires significant time and computational resources
Solution Approach 1:
The patent segments the weight adjustment process into two distinct phases: corrective weights that are rapidly adjusted during training, and synaptic weights that remain relatively stable. This segmentation allows the network to learn patterns quickly through corrective weight modifications without requiring extensive retraining of the entire network structure, thereby reducing training time while maintaining recognition accuracy
Solution Approach 2:
The patent applies preliminary action by pre-establishing the network structure and synaptic weights before training begins. The corrective weights are then rapidly adjusted during training to achieve pattern recognition without requiring full retraining of the network. This preliminary setup enables faster adaptation to new patterns while preserving previously learned knowledge
2Adaptability or versatility
If the size of a classical neural network is expanded to handle more complex tasks, then the network's capability increases, but full retraining is required which consumes additional resources
Solution Approach 1:
The patent implements dynamics by allowing the network to adapt its corrective weights dynamically when new elements are added, while maintaining stable synaptic weights. This dynamic adjustment mechanism enables the network to expand its capability to handle complex tasks without requiring full retraining, as the corrective weights can be rapidly reconfigured to accommodate new patterns and tasks
3Measurement precision
If traditional neural networks are trained extensively to avoid local minima, then recognition accuracy improves, but training resources and time increase significantly
Solution Approach 1:
The patent segments the weight system into corrective weights and synaptic weights, where only the corrective weights undergo extensive adjustment during training. This segmentation allows the network to achieve high recognition accuracy through focused adjustment of corrective weights without requiring computationally expensive full-network retraining, thereby reducing energy consumption while maintaining accuracy
Solution Approach 2:
The patent applies parameter changes by modifying only the corrective weight parameters during training while keeping synaptic weights relatively stable. This selective parameter adjustment enables the network to escape local minima and achieve high recognition accuracy without the computational burden of adjusting all network parameters, thus reducing energy consumption
Data Source
AI summary
A neural network includes a plurality of inputs for receiving input signals, and synapses connected to the inputs and having corrective weights established by a memory element that retains a respective weight value. The network additionally includes distributors. Each distributor is connected to one of the inputs for receiving the respective input signal and selects one or more corrective weights in correlation with the input value. The network also includes neurons. Each neuron has an output connected with at least one of the inputs via one synapse and generates a neuron sum by summing corrective weights selected from each synapse connected to the respective neuron. The output of each neuron provides the respective neuron sum to establish operational output signal of the network. A method of operating a neural network includes processing data thereby and using modified corrective weight values established by a separate analogous neural network during training thereof.