Dynamic Pruning Filter for Convolutional Neural Network Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional pruning techniques, both static and dynamic, face limitations in reducing computational load and memory usage of deep learning models while maintaining performance, particularly due to inefficiencies in path indexing and weight duplication.
Innovation Solution
A dynamic pruning filter method is introduced that generates an attention weight matrix based on feature maps and a mask matrix derived from convolution kernels, using global average pooling and binary step functions to optimize weight pruning, enabling efficient computation and memory reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If static pruning technique is used, then computational speed is improved, but model performance significantly decreases
Solution Approach 1:
The patent applies dynamic pruning by generating different sparse patterns for different input samples. The pruning filter is dynamically adjusted based on the input data characteristics, allowing the model to maintain high performance while achieving computational speedup. This resolves the contradiction by making the pruning strategy adaptive rather than fixed.
Solution Approach 2:
The patent changes the pruning parameters dynamically based on input samples. By adjusting the sparse pattern parameters according to input characteristics, the model maintains optimal performance across different inputs while still achieving significant computational acceleration compared to static pruning.
2Reliability
If dynamic pruning technique is used, then model performance is improved, but computational overhead increases due to path indexing and weight duplication
Solution Approach 1:
The patent extracts and removes redundant computational operations in dynamic pruning. By eliminating unnecessary path indexing and weight duplication operations, it reduces the computational overhead while maintaining the performance benefits of dynamic pruning. This selectively removes harmful operations while preserving useful ones.
Solution Approach 2:
The patent optimizes weight handling by avoiding unnecessary weight copying operations. Instead of duplicating weights for all input samples, it implements a more efficient weight management strategy that reduces memory operations and computational overhead while maintaining the dynamic pruning benefits.
3Reliability
If dynamic pruning technique is used, then model performance is improved, but memory usage reduction is limited
Solution Approach 1:
The patent implements efficient memory management by discarding redundant weight storage and recovering memory through optimized weight handling. By avoiding unnecessary weight duplication and implementing selective weight storage, it achieves significant memory usage reduction while maintaining dynamic pruning performance benefits.
Data Source
AI summary
There is an inference method using a dynamic pruning filter in a convolutional neural network model. The inference method comprises generating an attention weight matrix based on a feature map of at least one channel extracted from an input image; generating at least one mask matrix by referring to a convolution kernel included in the convolutional neural network model; and outputting the a dynamic pruning filter based on the operation of the attention weight matrix and the at least one mask matrix.