CNN Channel Pruning via Contribution Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural network (CNN) models face challenges due to high computational complexities and large model sizes, making them unsuitable for resource-constrained devices such as mobile phones, edge devices, and IoT devices.
Innovation Solution
An importance-based pruning system that automatically identifies and prunes away channels or filters of a CNN that are least important to the predictive accuracy, thereby reducing computational burden while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CNN models are used to achieve high predictive accuracy, then model performance is improved, but computational complexity and model size increase
Solution Approach 1:
The patent extracts and removes redundant or less important channels from the CNN model while retaining the most critical channels for predictive accuracy. This channel pruning approach reduces computational complexity by eliminating unnecessary processing paths while maintaining the model's essential functionality and accuracy performance.
Solution Approach 2:
The patent applies different importance metrics to different channels within the CNN model, evaluating each channel's contribution locally rather than applying a uniform reduction approach. This allows the model to maintain high quality in critical regions (important channels) while reducing complexity in less critical regions (redundant channels).
2Measurement precision
If CNN models are used to achieve high predictive accuracy, then model performance is improved, but model size increases
Solution Approach 1:
The patent extracts and removes redundant channels from the CNN model structure, directly reducing the number of parameters and model size. By identifying and eliminating channels that contribute minimally to predictive accuracy, the model achieves compression while maintaining performance.
Solution Approach 2:
The patent changes the parameter configuration of the CNN model by selectively retaining only the most important channels based on importance metrics. This parameter selection approach reduces the total number of model parameters and size while preserving the critical parameters necessary for high predictive accuracy.
3Productivity
If CNN models are pruned to reduce size, then model compression is improved, but predictive accuracy may deteriorate
Solution Approach 1:
The patent uses importance metrics to identify and retain the most critical parameters (channels) while removing less important ones. This selective parameter retention ensures that model compression is achieved without sacrificing the parameters most essential for predictive accuracy, thus maintaining performance while improving compression ratios.
Solution Approach 2:
The patent employs importance metrics that provide feedback on each channel's contribution to predictive accuracy. This feedback mechanism guides the pruning process by identifying which channels can be safely removed without degrading performance, enabling accurate model compression that preserves essential functionality.
Data Source
AI summary
Systems, methods, and other embodiments associated with contribution metric-based pruning of a neural network are described. In one embodiment, an example method includes accessing a trained neural network that has a plurality of channels. The neural network is to be evaluated for pruning of the channels. The example method may also include determining contribution metrics for the channels by measuring changes in error of the convolutional neural network with individual channels removed in turn. The contribution metrics are determined based at least in part on higher order analysis of the changes. And, the example method may also include pruning out of the convolutional neural network a set of the channels for which the contribution metrics do not satisfy a threshold.