Neural Network Filter Pruning for Faster Low-Memory Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) are over-parameterized, making them challenging to deploy on resource-constrained devices due to high memory requirements and slow inferencing speeds, with existing pruning methods either being complex or failing to improve performance.
Innovation Solution
A structured filter pruning method, Grad Square (GS), determines filter importance independently using gradient-based scoring, allowing for efficient compression without specialized hardware, preserving model performance by removing redundant filters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks are designed with large-scale parameters to achieve state-of-the-art performance, then inferencing accuracy is improved, but memory requirements increase and inferencing speed decreases
Solution Approach 1:
The patent extracts and removes redundant filters from the neural network through pruning. By identifying and eliminating filters with low importance scores (calculated via gradient-based methods), the system reduces the total number of parameters while maintaining the essential representational capacity of the network, thus reducing memory requirements without significantly compromising accuracy
Solution Approach 2:
The patent changes the parameter configuration by transitioning from a dense filter structure to a pruned filter structure. Through parameter-freezing during pruning and subsequent fine-tuning, the system adapts the remaining filters to compensate for removed parameters, achieving efficient compression of the model size
2Measurement precision
If deep neural networks are designed with large-scale parameters to achieve state-of-the-art performance, then inferencing accuracy is improved, but inferencing speed decreases
Solution Approach 1:
The patent extracts and removes redundant computational operations through filter pruning. By eliminating filters that contribute minimally to the network's representational capacity, the system reduces the total number of floating-point operations required during inferencing, thereby increasing inferencing speed while preserving accuracy through careful selection of filters to remove
Solution Approach 2:
The patent segments the filter importance assessment into independent components by calculating importance scores for each filter based on gradient magnitudes. This segmentation allows for systematic identification and removal of specific redundant filters without affecting the entire network structure, enabling optimized inferencing speed while maintaining accuracy
3Quantity of substance
If existing pruning methods are applied to reduce model complexity, then memory requirements are reduced, but training complexity increases
Solution Approach 1:
The patent performs preliminary action by calculating filter importance scores using gradient-based methods before actually removing filters. By pre-computing importance metrics (such as L1 or L2 norms of gradients) and ranking filters in advance, the system simplifies the pruning process to a straightforward threshold-based removal operation, reducing training complexity while achieving memory reduction
Solution Approach 2:
The patent applies self-service by using the network's own gradient computations during normal training to automatically identify important filters. The gradient-based importance scoring leverages existing computational operations to serve the dual purpose of training and pruning, eliminating the need for separate complex analysis procedures and reducing overall training complexity
4Quantity of substance
If existing pruning methods are applied to reduce model complexity, then memory requirements are reduced, but inferencing speed improvement is insufficient
Solution Approach 1:
The patent changes the parameter configuration by implementing structured pruning that removes entire filters rather than individual weights. This creates a more regular, compressed network structure with reduced computational graph complexity, which translates to faster inferencing speeds on standard hardware while achieving significant memory reduction through filter elimination
Data Source
AI summary
Techniques are provided for compressing deep neural networks using a structured filter pruning method that is extensible and effective. According to an embodiment, a computer-implemented method comprises determining, by a system operatively coupled to a processor, importance scores for filters of layers of a neural network model previously trained until convergence for an inferencing task on a training dataset. The method further comprises removing, by the system, a subset of the filters from one or more layers of the layers based on the importance scores associated with the subset failing to satisfy a threshold importance score value. The method further comprises converting, by the system, the neural network model into a compressed neural network model with the subset of the filters removed.