Neural Network Training Using Segmented Learning Stages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural network (ANN) training methods, particularly in deep neural networks, face inefficiencies due to nested iterative processes and the 'curse of dimensionality', leading to slow learning times and excessive computational complexity.
Innovation Solution
The proposed solution divides the learning process into three stages: self-organization, modified error correction, and back propagation, using the Modified Generalized Hebbian Algorithm and a modified method of error correction to reduce complexity and learning time, and introduces a cost of connection measure to control topology and weight adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional back propagation method is used for training deep neural networks, then learning accuracy can be achieved, but learning time complexity becomes exponential and computational complexity becomes excessive
Solution Approach 1:
The patent segments the training process into three distinct stages: self-organization stage (unsupervised learning for hidden layers), modified error correction stage (supervised learning for output layer), and back propagation stage (fine-tuning). This segmentation allows each stage to use the most appropriate algorithm, avoiding the exponential complexity of traditional back propagation while maintaining learning accuracy.
Solution Approach 2:
The patent applies preliminary action by performing unsupervised self-organization learning in the first stage to pre-train hidden layers and extract essential features before proceeding to supervised learning. This preliminary feature extraction simplifies the subsequent supervised learning task and reduces overall computational complexity.
2Measurement precision
If traditional back propagation method is used for training deep neural networks, then learning accuracy can be achieved, but computational complexity becomes excessive
Solution Approach 1:
The patent segments the training process into three distinct stages: self-organization stage (unsupervised learning for hidden layers), modified error correction stage (supervised learning for output layer), and back propagation stage (fine-tuning). This segmentation allows each stage to use the most appropriate algorithm, avoiding the exponential complexity of traditional back propagation while maintaining learning accuracy.
Solution Approach 2:
The patent extracts and removes the computationally expensive nested iterative processes from the training algorithm by replacing them with unsupervised self-organization learning in the first stage. This extraction eliminates the curse of dimensionality while preserving the essential learning functionality.
3Reliability
If nested iterative processes are used in learning algorithms, then comprehensive error correction can be performed, but learning time increases exponentially
Solution Approach 1:
The patent segments the training process into three distinct stages: self-organization stage (unsupervised learning for hidden layers), modified error correction stage (supervised learning for output layer), and back propagation stage (fine-tuning). This segmentation allows each stage to use the most appropriate algorithm, avoiding the exponential complexity of traditional back propagation while maintaining learning accuracy.
Solution Approach 2:
The patent implements continuous useful action by processing all training samples through the self-organization stage in a single pass without nested iterations, then proceeding continuously to the modified error correction stage. This eliminates redundant iterative computations while maintaining comprehensive error correction through the distributed error correction mechanism.
4Measurement precision
If supervised learning is applied to all layers including hidden layers, then learning accuracy improves, but learning time and computational resources increase significantly
Solution Approach 1:
The patent applies local quality by using different learning algorithms for different layers: unsupervised self-organization learning for hidden layers (where feature extraction is the primary goal) and supervised learning for the output layer (where accurate classification is the primary goal). This localized algorithm selection optimizes both accuracy and efficiency for each layer's specific function.
Solution Approach 2:
The patent applies preliminary action by performing unsupervised self-organization learning in the first stage to pre-train hidden layers and extract essential features before proceeding to supervised learning. This preliminary feature extraction simplifies the subsequent supervised learning task and reduces overall computational complexity.
Data Source
AI summary
A modified model of the Rosenblatt perceptron and learning method leading to a significant increase in the intelligence and learning speed of the Rosenblatt perceptron itself, as well as arbitrary neural network topologies based on it, based on the modified error correction method and the modified “Back propagation” algorithm. To increase the manageability of the perceptron topology, as well as the processes of recognition and learning, the cost of connection is added at the synapses in addition to the weighting factor. To increase the speed of the perceptron learning, first, at least at the initial stage of learning, the process is moved from nested recursive iterations of learning on different layers of synapses of perceptron to successive learning for different layers; and, second, the process is moved from the constant rate of learning parameter, which requires a sequence of small iterations when adjusting the weights, to the functional parameter of the learning speed, correcting the weight so that the error for the current image is compensated completely in one step.