Passive CNN Filter Pruning for Embedded Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As convolutional neural networks (CNNs) grow deeper, the computational costs of inference increase, posing challenges for embedded sensors and mobile devices with limited resources, and existing methods of model compression through weight pruning do not effectively reduce computation time due to sparse models lacking practical optimizations.
Innovation Solution
The method involves actively pruning filters in CNNs by multiplying weights with an attrition factor less than one during training, removing weights and associated kernels when their contribution falls below a threshold, thereby reducing computational costs without increasing sparsity, and correlating the number of pruned filters with computational acceleration through reduced matrix multiplications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If model compression is achieved by pruning weights with small magnitudes, then storage costs are reduced, but computation time is not reduced because the majority of removed parameters are from fully connected layers where computation cost is low
Solution Approach 1:
The patent applies different pruning strategies to different parts of the neural network. Specifically, it prunes filters (entire sets of weights) in convolutional layers rather than individual weights, and targets different layers with different pruning approaches. This local differentiation ensures that computation-intensive convolutional operations are reduced while maintaining model accuracy.
Solution Approach 2:
The patent segments the neural network into different layers (convolutional layers and fully connected layers) and applies pruning selectively. It identifies and prunes specific filters in convolutional layers based on their contribution to the output, while using different criteria for fully connected layers. This segmentation allows targeted reduction of computational cost in the most impactful areas.
2Measurement precision
If deeper convolutional neural networks are used, then model accuracy is improved, but computational cost of inference increases
Solution Approach 1:
The patent extracts and removes unnecessary filters from the neural network that do not contribute significantly to the output. By identifying filters with low contribution metrics and pruning them entirely, the network depth and complexity are reduced while maintaining the accuracy provided by the remaining essential filters. This extraction approach allows deeper networks to be pruned back to optimal complexity.
Solution Approach 2:
The patent changes the number of filters (a key parameter) in convolutional layers dynamically during training. It starts with a full set of filters and progressively prunes them based on their contribution, effectively changing the network architecture parameter to balance accuracy and computational cost. This parameter adjustment allows the network to maintain accuracy while reducing power consumption.
3Quantity of substance
If pruning creates sparse models, then storage is reduced, but computations become less practical due to lack of optimizations
Solution Approach 1:
Instead of creating sparse models by removing individual weights (which creates irregular sparsity patterns), the patent inverts the approach by removing entire filters, resulting in dense subnetworks. This inversion creates a more regular and practical structure that is easier to optimize and implement, while still achieving significant compression and storage reduction.
Data Source
AI summary
Methods and systems of training a neural network includes training a neural network based on training data. Weights of a layer of the neural network are multiplied by an attrition factor. A block of weights is pruned from the layer if the block of weights in the layer has a contribution to an output of the layer that is below a threshold.