Group-wise DNN Filter Pruning via Graph Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks (DNNs) are too large to fit in systems with limited computing resources, such as mobile devices, due to their high memory and energy consumption, and current filter pruning methods rely on heuristic approaches that are time-intensive and indiscriminate.
Innovation Solution
A learnable framework that expresses the DNN workload as a sequential graph representation and uses a trained graph neural network to determine group-wise pruning ratios, allowing for efficient filter pruning on a per-group basis, reducing the number of filter groups and achieving higher sparsity levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If filter pruning methods are applied to reduce DNN size, then the model can fit in systems with limited computing resources, but the pruning process becomes time-intensive and indiscriminate
Solution Approach 1:
The patent segments the DNN into multiple groups of filters, where each group is processed independently. This segmentation allows the pruning process to handle large networks by dividing them into manageable units, reducing the time required for indiscriminate pruning while maintaining the ability to reduce overall model size.
Solution Approach 2:
The patent introduces learnable parameters that enable the model to automatically determine optimal pruning ratios for each filter group. By changing from fixed heuristic pruning to learnable parameter-based pruning, the system reduces time consumption while achieving more targeted and efficient filter removal.
2Quantity of substance
If filter pruning is applied to reduce DNN size, then memory and energy consumption are reduced, but the accuracy of the model may be compromised
Solution Approach 1:
The patent applies local quality by allowing different pruning ratios for different filter groups based on their specific characteristics and importance. This enables the model to remove less critical filters while preserving important ones, thereby reducing the number of filters while maintaining model accuracy.
Solution Approach 2:
The patent incorporates feedback mechanisms where the pruning process is guided by learnable parameters that are updated based on model performance. This feedback loop ensures that pruning operations maintain model accuracy by adjusting the pruning ratios dynamically based on actual model behavior and performance metrics.
3Ease of manufacture
If heuristic-based pruning is used, then the pruning process is simple to implement, but it is time-intensive and cannot adapt to different DNN architectures
Solution Approach 1:
The patent creates a universal pruning framework that can be applied to different DNN architectures through learnable parameters. This framework maintains ease of implementation while significantly improving productivity by adapting to various network structures without requiring architecture-specific pruning methods.
Solution Approach 2:
The patent introduces dynamic pruning ratios that can adapt during the training process. Instead of using static heuristic thresholds, the system dynamically adjusts pruning parameters based on learnable models, improving pruning efficiency while maintaining implementation simplicity through a unified framework.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus is provided to compress DNNs using filter pruning on a per-group basis. For example, the apparatus accesses a trained DNN that includes a plurality of layers. The apparatus generates a sequential graph representation of the plurality of layers. The sequential graph representation includes a sequence of nodes. Each node is a graph representation of a layer. The apparatus clusters the layers into layer groups. A layer group includes one or more layers. The apparatus determines a pruning ratio for a layer group and prunes the filters of the layers in the layer group based on the pruning ratio. The apparatus may cluster the layers and determine the pruning ratio by using a GNN. The apparatus generates compressed layers from the layers in the layer group through the filter pruning process. The apparatus further updates the DNN by replacing the layers in the layer group with the compressed layers.