Pruning Layer Reduces Convolution Channel Count in Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep artificial neural networks exhibit high computational complexity and require significant data storage, posing a challenge for designers seeking to reduce these requirements.
Innovation Solution
A compression method involving the use of pruning layers with weighting values to identify and delete redundancy channels in convolution layers, thereby generating a compressed artificial neural network that reduces computational complexity and data storage needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep artificial neural networks are used to solve visual-related application problems, then the problem-solving capability is improved, but the computational complexity and data storage space requirements increase
Solution Approach 1:
The patent extracts and removes redundant channels from convolution layers in the neural network. By identifying and eliminating channels that contribute minimally to the output (those with small weighting values in the pruning layer), the system reduces computational complexity and data storage requirements while preserving the essential problem-solving capability of the network.
Solution Approach 2:
The patent changes the parameter of channel quantity in convolution layers by dynamically pruning channels based on their weighting values. The pruning layer assigns weighting values to each channel, and channels below a certain threshold are removed. This parameter change allows the network to adapt its complexity to the actual task requirements, reducing computational burden while maintaining effectiveness.
2Reliability
If deep artificial neural networks are used to solve visual-related application problems, then the problem-solving capability is improved, but the data storage space requirements increase
Solution Approach 1:
The patent extracts and removes redundant channels from convolution layers in the neural network. By identifying and eliminating channels that contribute minimally to the output (those with small weighting values in the pruning layer), the system reduces computational complexity and data storage requirements while preserving the essential problem-solving capability of the network.
3Productivity
If the number of channels in convolution layers is reduced to decrease computational complexity, then the computational efficiency is improved, but the training error rate may increase
Solution Approach 1:
The patent incorporates feedback mechanisms to monitor the training error rate during the pruning process. The system calculates a moving average of the training error rate and compares it with a threshold. When the error rate exceeds the threshold, the system activates a restoring procedure to recover from over-pruning. This feedback loop ensures that computational efficiency is improved while maintaining acceptable training performance.
Solution Approach 2:
The patent prepares compensatory measures in advance by setting up a restoring procedure that can be activated when the training error rate exceeds a threshold. This prior cushioning ensures that if over-pruning occurs and degrades performance, the system can recover by restoring previously pruned channels, thus preventing permanent loss of accuracy while still achieving computational efficiency gains.
Data Source
AI summary
An electronic apparatus and a compression method for an artificial neural network are provided. The compression method is adapted for the artificial neural network with a plurality of convolution layers. The compression method includes: setting a first pruning layer for coupling the first pruning layer to Lth convolution layer, where the first pruning layer has a plurality of first weighting values and each of the first weighting values corresponds to each of a plurality of channels of the Lth convolution layer; tuning the first weighting values, selecting a part of the channels of the Lth convolution layer to be at least one first redundancy channel according to the first weighting values, and generating a compressed Lth convolution layer by deleting the at least one first redundancy channel; and removing the first pruning layer, and generating a first compressed artificial neural network.