AI Model Compression via Structured Channel Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing model compression methodologies for AI models are inefficient in terms of time and resources, particularly as model sizes increase, and they lack effective global pruning strategies.
Innovation Solution
A model compression method that combines unstructured pruning with structured pruning, using criteria and sparsity to determine filters for pruning, thereby generating a compressed AI model that can be optimized for equipment-friendly performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If unstructured pruning is applied to mask values with zeroes, then model compression is achieved, but actual operation amount for matrix multiplication is not reduced and acceleration library or dedicated hardware is required
Solution Approach 1:
The patent segments the model into channels and applies pruning at the channel level rather than individual parameter level. This structured approach allows standard hardware to efficiently handle the pruned model without requiring specialized acceleration libraries, as the pruning pattern aligns with natural hardware parallelism in channel processing
Solution Approach 2:
Instead of masking values with zeroes as in traditional unstructured pruning, the patent inverts the approach by directly removing entire channels (filters) from the model. This eliminates the need for zero-value storage and masking operations, achieving compression through actual removal of model components rather than logical masking
2Volume of stationary object
If existing structured pruning methodologies are used, then some compression is achieved, but they are very inefficient in terms of time and resources as model size increases
Solution Approach 1:
The patent performs preliminary channel importance evaluation using activation statistics from a single forward pass before pruning. This preliminary assessment identifies which channels to remove without requiring iterative pruning and verification, significantly reducing the time and computational resources needed compared to existing methodologies that repeatedly evaluate and verify pruning decisions
Solution Approach 2:
The patent changes the pruning parameter from traditional L1/L2 norm-based importance metrics to activation-based importance metrics. This parameter change enables more accurate identification of redundant channels while requiring fewer computational resources, as activation statistics can be computed efficiently during a single forward pass without additional training iterations
3Volume of stationary object
If random pruning or layer-by-layer pruning is performed, then some compression is achieved, but global pruning effectiveness is lost and verification of model performance impact is required
Solution Approach 1:
The patent applies a universal channel pruning criterion across all layers of the model based on activation importance. This global criterion simultaneously evaluates channels in different layers using the same activation-based metric, enabling coordinated pruning decisions that maintain model performance while achieving effective global compression without layer-by-layer verification
Data Source
AI summary
Disclosed is a model compression method and system for compressing a model for optimizing to an equipment-friendly model. A model compression method may include acquiring criteria and sparsity for each filter of a model to which unstructured pruning is already applied, determining a filter for applying structured pruning among filters of the model based on the criteria and the sparsity, and applying the structured pruning to the model based on the determined filter.