Sparse Neuronal Network Pruning for Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks require a large number of model neurons, leading to increased power consumption and computational complexity, which hinders scalability and efficiency in applications such as image and pattern recognition.
Innovation Solution
A method and apparatus for selecting a reduced number of model neurons in a neural network by generating a sparse set of non-zero decoding vectors associated with synapses between neuron layers, allowing the network to operate only with selected model neurons, thereby reducing the number of neurons and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of model neurons are used in the neural network, then the computational accuracy and pattern recognition capability are improved, but the power consumption and device complexity increase
Solution Approach 1:
The patent extracts and removes redundant or less significant neurons from the neural network while retaining the essential neurons needed for accurate computation. This is achieved through techniques such as neuron importance scoring, pruning algorithms, and sparse connectivity patterns that eliminate unnecessary computational units, thereby reducing power consumption while maintaining computational accuracy.
Solution Approach 2:
The patent applies local quality by creating heterogeneous neuron populations where different neurons have different levels of activity, connectivity, or importance. Rather than uniformly treating all neurons, the system identifies and enhances critical neurons while allowing less critical ones to be reduced or removed, optimizing the balance between accuracy and power consumption at the local neuron level.
2Measurement precision
If a large number of model neurons are used in the neural network, then the computational accuracy is improved, but the device complexity increases
Solution Approach 1:
The patent extracts and removes redundant or less significant neurons from the neural network while retaining the essential neurons needed for accurate computation. This is achieved through techniques such as neuron importance scoring, pruning algorithms, and sparse connectivity patterns that eliminate unnecessary computational units, thereby reducing power consumption while maintaining computational accuracy.
Solution Approach 2:
The patent applies local quality by creating heterogeneous neuron populations where different neurons have different levels of activity, connectivity, or importance. Rather than uniformly treating all neurons, the system identifies and enhances critical neurons while allowing less critical ones to be reduced or removed, optimizing the balance between accuracy and power consumption at the local neuron level.
3Use of energy by stationary object
If the number of model neurons is reduced, then the power consumption and device complexity are decreased, but the computational accuracy may deteriorate
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network with a complete set of neurons to achieve high computational accuracy, then performing a second phase of neuron selection and pruning to reduce the network size. This preliminary training ensures that the essential computational patterns are learned before reducing the neuron count, thereby maintaining accuracy while reducing power consumption in the deployed network.
Solution Approach 2:
The patent uses feedback mechanisms to monitor computational accuracy during the neuron reduction process. By continuously evaluating network performance and adjusting which neurons are retained or removed, the system ensures that accuracy thresholds are maintained while achieving power consumption reduction goals. Feedback from performance metrics guides the iterative pruning process.
Data Source
AI summary
A method for selecting a reduced number of model neurons in a neural network includes generating a first sparse set of non-zero decoding vectors. Each of the decoding vector is associated with a synapse between a first neuron layer and a second neuron layer. The method further includes implementing the neural network only with selected model neurons in the first neuron layer associated with the non-zero decoding vectors.