DNN Weight Pruning with Kernel Sparsity for Mobile Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural network (DNN) models are challenging to execute in real-time on mobile devices due to their intensive computation and high memory storage requirements, with current compression techniques either compromising accuracy or not being hardware-friendly.
Innovation Solution
Introduce a new sparsity dimension through intra- and inter-convolution kernel pruning, combined with a compiler-assisted DNN inference framework, utilizing sparse convolution patterns and connectivity pruning to achieve high accuracy and regularity, enabling real-time execution on mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If non-structured weight pruning is applied to reduce model size, then compression rate is improved, but processing throughput deteriorates due to complex workload and stalls in highly parallel architectures
Solution Approach 1:
The patent applies local quality by differentiating between intra-kernel sparsity (non-structured pruning within individual convolution kernels) and inter-kernel sparsity (structured pruning across kernel groups). This localized approach allows fine-grained compression where needed while maintaining regularity in critical regions, resolving the contradiction between high compression and sustained throughput in parallel architectures.
Solution Approach 2:
The patent introduces a new sparsity dimension by combining both intra-convolution kernel sparsity and inter-convolution kernel sparsity. This multi-dimensional sparsity approach goes beyond traditional single-level pruning, enabling simultaneous achievement of high compression rate and maintained throughput by exploiting sparsity at multiple granularities.
2Ease of manufacture
If structured weight pruning is applied to improve hardware friendliness and eliminate index storage requirements, then ease of manufacture is improved, but accuracy deteriorates more significantly
Solution Approach 1:
The patent applies partial action by selectively applying structured pruning at the inter-kernel level rather than uniformly across all weights. This partial structured pruning maintains accuracy by preserving critical intra-kernel weight relationships while still achieving hardware friendliness through inter-kernel sparsity patterns.
Solution Approach 2:
The patent segments the weight matrix into intra-kernel and inter-kernel components, applying different sparsity strategies to each segment. This segmentation allows the system to maintain accuracy in the intra-kernel segment while achieving hardware optimization in the inter-kernel segment.
3Measurement precision
If large-scale DNN models are deployed to achieve high accuracy, then measurement precision is improved, but execution time deteriorates making real-time inference unachievable
Solution Approach 1:
The patent applies parameter changes by transforming the sparsity pattern parameters of the DNN model through combined intra- and inter-convolution kernel pruning. This parameter transformation enables the model to maintain high accuracy while reducing computational complexity, achieving real-time inference performance.
Data Source
AI summary
A computer-implemented method is disclosed for compressing a deep neural network (DNN) model by DNN weight pruning to accelerate DNN inference on mobile devices. The method includes the steps of (a) performing an intra-convolution kernel pruning of the DNN model wherein a fixed number of weights are pruned in each convolution kernel of the DNN model to generate sparse convolution patterns; (b) performing inter-convolution kernel pruning of the DNN model to generate connectivity sparsity, wherein inter-convolution kernel pruning comprises cutting connections between given input and output channels of the DNN model to remove corresponding kernels; and (c) training the DNN model compressed in steps (a) and (b).