Layer-Wise Adaptive Channel Pruning for Deep Learning Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep-learning model inference computation techniques face challenges in accelerating computation while maintaining accuracy, particularly in resource allocation within computing clusters, where channel pruning methods focus on minimizing accuracy decrease but often prioritize reducing a large number of channels with small acceleration effects over fewer channels with significant computation reduction.
Innovation Solution
A layer-wise adaptive channel pruning method that profiles pruning sensitivity, compares memory occupancy reduction and computation amount reduction influences on throughput, and performs channel pruning based on model layer-wise resource memory occupancy or computation characteristics to maximize batch size and throughput while ensuring model analysis accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If channel pruning removes a large number of channels to minimize accuracy decrease, then model accuracy is preserved, but computation acceleration effect is reduced because many removed channels have small computation reduction impact
Solution Approach 1:
The patent applies local quality by differentiating channels based on their individual computation characteristics and pruning sensitivity. Instead of uniform pruning, it identifies and prunes channels with high computation reduction impact while preserving channels critical for accuracy, achieving both accuracy maintenance and computation acceleration through localized differentiation of channel treatment
Solution Approach 2:
The patent changes the pruning parameter from simple channel count reduction to layered adaptive pruning based on computation characteristics. By adjusting pruning intensity and selection criteria at different network layers according to their specific computation profiles, it optimizes the balance between accuracy preservation and acceleration effect
2Productivity
If channel pruning reduces computation amount significantly, then inference speed increases, but resource memory occupancy may increase due to larger batch size requirements
Solution Approach 1:
The patent applies dynamics by adaptively adjusting batch size based on the pruned model's specific computation characteristics. Instead of using a fixed batch size, it dynamically determines the optimal batch size that maximizes throughput while keeping memory occupancy within acceptable limits, allowing flexible adaptation to different pruning configurations
3Productivity
If weight pruning removes individual parameters to achieve computation acceleration, then parameter size reduces, but it requires sparse matrix computation support software or hardware which increases system complexity
Solution Approach 1:
The patent extracts the complexity requirement by choosing channel pruning over weight pruning. Channel pruning removes entire channels rather than individual parameters, which can be implemented using standard dense matrix operations without requiring sparse matrix computation support, thereby achieving computation acceleration without the associated software/hardware complexity
Data Source
AI summary
A control method and system based on layer-wise adaptive channel pruning are provided. The control method includes: profiling a layer-wise pruning sensitivity of an original deep-learning model; comparing an influence of a resource memory occupancy reduction on a throughput of an accelerator resource with an influence of a computation amount reduction on the throughput of the accelerator resource; performing, based on the comparing, a channel pruning based on a model layer-wise resource memory occupancy characteristic of the original deep-learning model or a model layer-wise computation amount characteristic of the original deep-learning model; in response to the channel-pruned model satisfying a certain model analysis accuracy level, determining a batch size for the accelerator resource; and in response to a throughput of the channel-pruned model based on the determined batch size being greater than a throughput of the original deep-learning model, employing the channel-pruned model in the deep-learning model computation acceleration.