PatDNN Pattern-Based Pruning for Mobile DNN Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for achieving real-time deep neural network (DNN) execution on mobile devices face challenges due to limited computing resources, with existing DNN model compression techniques either sacrificing accuracy or not fully utilizing hardware potential, particularly in non-structured and structured pruning approaches.
Innovation Solution
The introduction of a novel end-to-end mobile DNN acceleration framework, PatDNN, which employs pattern-based pruning methods combined with compiler optimizations to achieve high accuracy and execution efficiency, leveraging kernel pattern and connectivity pruning to bridge the gap between non-structured and structured pruning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If general non-structured weight pruning is applied to achieve high compression ratio, then model size is reduced, but processing throughput deteriorates due to loss of parallelism
Solution Approach 1:
The patent segments the weight pruning process into two distinct stages: (1) non-structured pruning at the weight level to achieve high compression ratio, and (2) structured pruning at the filter/channel level to restore hardware efficiency. This segmentation allows each pruning type to address its strength while compensating for its weakness through the combined effect of both stages.
Solution Approach 2:
The patent extracts and removes pruned weights and filters from the DNN model in a systematic two-step process. First, individual weights are pruned based on importance metrics; then, entire filters or channels that have become redundant are extracted and removed. This extraction process creates a sparsified model that maintains high compression while enabling hardware-friendly structured operations.
2Productivity
If structured pruning (filter and channel pruning) is applied to generate hardware-friendly models, then processing throughput is improved, but model compression ratio deteriorates due to coarser granularity
Solution Approach 1:
The patent implements a dynamic, two-stage pruning strategy where the pruning granularity adapts through different phases. In the first stage, fine-grained weight-level pruning is applied dynamically to achieve maximum compression. In the second stage, the model is reorganized into a structured format where filter-level pruning can be applied to restore hardware efficiency. This dynamic adaptation of pruning granularity resolves the contradiction between compression ratio and throughput.
Solution Approach 2:
The patent transitions from a single-dimension pruning approach to a two-dimensional pruning space. The first dimension is weight-level pruning for compression, and the second dimension is filter/channel-level pruning for structure. By operating in this expanded pruning space, the method simultaneously achieves high compression ratios and hardware-friendly structures that improve processing throughput.
3Ease of manufacture
If iterative and heuristic methods are used for DNN model compression, then implementation simplicity is maintained, but compression rate becomes limited and non-uniform
Solution Approach 1:
The patent incorporates feedback mechanisms in both pruning stages. In non-structured pruning, importance metrics are computed based on feedback from model performance and weight utilization patterns. In structured pruning, feedback from accuracy evaluation guides the selection of filters and channels to prune. This feedback-driven approach enables the method to achieve high and uniform compression rates while maintaining implementation simplicity through automated decision-making.
Solution Approach 2:
The patent systematically changes multiple parameters throughout the two-stage pruning process, including pruning ratios, importance threshold values, and filter selection criteria. These parameter adjustments are made based on feedback from model evaluation at each stage, allowing the method to achieve optimal compression rates while maintaining simplicity through automated parameter tuning rather than complex heuristic rules.
Data Source
AI summary
PatDNN is an end-to-end framework to achieve real-time DNN execution on mobile devices. PatDNN includes two stages: a pattern-based pruning stage based on extended ADMM solution framework, and an optimized execution code generation stage including a high-level, fine-grained DNN layerwise representation and a set of architecture-aware optimizations. This design allows PatDNN to benefit from both high accuracy and hardware efficiency.