Neural Network Training via Hierarchical Neuron Addition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network training methods result in overly complex models with a large number of neurons, leading to increased training time, instability, and a high risk of over-training, which limits their implementation in mobile systems and reduces classification accuracy.
Innovation Solution
A method for training neural networks that automatically generates a hierarchical structure with a minimal number of neurons, using GPU-adapted algorithms for iterative parallel computation, reducing the search interval with each iteration, and adding new neurons only when necessary to improve accuracy and prevent over-training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional neural network training methods are used to achieve classification capability, then the network can learn from data, but the model becomes overly complex with a large number of neurons, leading to increased training time and risk of over-training
Solution Approach 1:
The patent applies segmentation by dividing the neural network into hierarchical levels (input layer, hidden layers with different functionalities, output layer) where each segment performs specific tasks. The network is organized into functional modules including correlation neurons, differentiation neurons, and integration neurons, allowing complex classification to be achieved through coordinated simple units rather than requiring a large monolithic network
Solution Approach 2:
The patent extracts and removes unnecessary neurons and connections from the network during training. The pruning mechanism identifies and eliminates redundant neural elements that do not contribute to classification performance, thereby reducing model complexity while maintaining accuracy. This is achieved through monitoring neuron activation patterns and removing those with minimal impact on output
2Reliability
If conventional neural network training methods are used to ensure model coverage, then all data patterns are considered, but training time increases significantly
Solution Approach 1:
The patent applies preliminary action through pre-training procedures where the network is initially trained on simplified or representative subsets of data to establish basic classification capabilities. This preliminary training phase allows the network to learn fundamental patterns before fine-tuning on complete datasets, significantly reducing overall training time while maintaining comprehensive model coverage
Solution Approach 2:
The patent implements partial action by training the network on strategically selected subsets of data that capture the essential classification boundaries. Rather than processing every data point exhaustively, the method uses representative samples and adaptive sampling techniques to achieve sufficient model coverage with reduced computational effort
3Measurement precision
If conventional neural network training methods are used to achieve accuracy, then detailed learning occurs, but the network becomes prone to over-training
Solution Approach 1:
The patent applies feedback mechanisms through continuous monitoring of validation performance during training. When the network's performance on validation data begins to deteriorate while training error continues to decrease, the training process is automatically stopped or adjusted. This feedback loop prevents over-training by detecting the point where additional learning harms generalization capability
Solution Approach 2:
The patent implements dynamic adjustment of training parameters including learning rate, batch size, and network architecture during the training process. The network structure and training conditions adapt in real-time based on performance metrics, allowing the system to optimize for precision when needed while maintaining resistance to over-training through flexible parameter modification
4Reliability
If conventional neural network training methods are used to build comprehensive models, then all features are processed, but the resulting model is too heavy for mobile systems
Solution Approach 1:
The patent extracts and removes redundant features, neurons, and connections from the neural network model. Through systematic pruning and feature selection, the model retains only the essential elements necessary for reliable classification, significantly reducing model size while maintaining classification reliability. This extraction process eliminates computational overhead and memory requirements
Solution Approach 2:
The patent applies local quality by assigning different levels of complexity to different parts of the network based on their importance. Critical features and neurons that contribute most to classification reliability are preserved with high fidelity, while less important elements are simplified or removed. This selective approach optimizes the balance between model size and reliability for mobile deployment
Data Source
AI summary
Method of training a neural network, including receiving sets of digital attributes representing multidimensional regression at inputs; expanding the network by adding neurons and defining their activation functions and interconnections; any neuron of the neural network is capable of being directly connected to any other neuron; (i) when a training speed falls below predefined threshold, and accuracy does not improve, identifying a neuron with highest error value; (ii) adding a neuron directly between the identified neuron and a corresponding output; (iii) setting only a connection coefficient between the added neuron and the identified neuron to zero before it is modified by the training, while other coefficients of the added neuron are set the same as coefficients of the identified neuron, before they are modified. After at least one iteration, either (iv) finishing the training of the neural network or (v) continuing to train the network to reach a predefined depth.