Neural Network Pruning via Layer-Specific Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural networks require significant computational power for inference operations due to the large number of computations needed, which can be inefficient and resource-intensive.
Innovation Solution
A method and device for pruning neural networks by initializing threshold values for each layer, adjusting these thresholds to classify neurons as active or inactive, and adjusting the weights of synapses to reduce computational complexity while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a conventional neural network is used for inference operations, then comprehensive computation coverage is achieved, but computational power requirement becomes very large
Solution Approach 1:
The patent extracts and removes inactive neurons from the neural network based on threshold comparison. By identifying neurons whose values fall below a predetermined threshold and setting their weights to zero, the system eliminates unnecessary computational elements while preserving the functional integrity of the network, thus reducing computational power requirements without sacrificing computation coverage.
Solution Approach 2:
The patent applies different threshold values to different layers of the neural network, creating local quality variations. Each layer can have its own customized threshold, allowing selective pruning tailored to the specific characteristics of each layer. This enables optimized computation coverage in each layer while collectively reducing overall computational power requirements.
2Device complexity
If threshold values are adjusted to prune neurons, then computational complexity is reduced, but network accuracy may deteriorate
Solution Approach 1:
The patent employs feedback mechanisms where threshold values are dynamically adjusted based on the state of neuron activations. By monitoring which neurons are activated and which remain inactive, the system refines threshold settings to maintain accurate inference results while achieving the desired level of pruning. This feedback loop ensures that computational complexity is reduced without compromising network accuracy.
Solution Approach 2:
The patent changes the threshold parameter to control the pruning process. By adjusting the threshold value, the system can control the degree of neuron elimination, finding an optimal balance between computational complexity reduction and accuracy maintenance. Different threshold values can be experimented with to achieve the desired trade-off between these two parameters.
Data Source
AI summary
A method for pruning a neural network includes initializing a plurality of threshold values respectively corresponding to a plurality of layers included in the neural network; selecting one of the plurality of layers; adjusting the threshold value of the selected layer; and adjusting a plurality of weights respectively corresponding to a plurality of synapses included in the neural network.