Edge DNN Structured Pruning at Initialization for Rapid Deployment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pruning methods for deep neural networks (DNNs) fail to simultaneously achieve high accuracy, reduced model size, and rapid deployment on edge devices due to computational, memory, and energy constraints, with unstructured pruning maintaining accuracy but not improving runtime, and structured pruning improving performance but reducing accuracy.
Innovation Solution
A method involving structured pruning at initialization (SPaI) that evaluates layer sensitivity and applies structured pruning only to resilient layers, followed by reinitialization, to generate a pruned model suitable for edge computing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If unstructured pruning is applied to maintain model accuracy, then model accuracy is preserved, but runtime performance is not improved
Solution Approach 1:
The patent segments the DNN model into different layers and applies selective structured pruning to specific layers while preserving others, enabling runtime performance improvement without sacrificing overall model accuracy. The model is divided into pruned layers (for speed) and unpruned layers (for accuracy preservation).
Solution Approach 2:
Different pruning strategies are applied to different parts of the network. The patent applies structured pruning to certain layers while maintaining unpruned structures in other layers, creating local variations in model density that optimize both runtime performance and accuracy for specific network regions.
2Productivity
If structured pruning is applied to improve runtime performance, then speed is improved, but model accuracy is reduced
Solution Approach 1:
The patent divides the neural network into segments where structured pruning is applied to some layers and not others. This segmentation allows the system to achieve runtime performance benefits from pruned layers while maintaining accuracy through unpruned layers.
Solution Approach 2:
Instead of applying structured pruning to the entire model (excessive action that would harm accuracy), the patent applies it partially to only certain layers. This partial application provides enough compression for runtime improvement while leaving sufficient capacity in unpruned layers to maintain accuracy.
3Quantity of substance
If neural architecture search is used to find suitable compressed models, then model compression is achieved, but substantial compute resources and time are consumed
Solution Approach 1:
The patent performs preliminary structured pruning at model initialization before training begins, rather than conducting extensive neural architecture search during or after training. This preliminary action dramatically reduces model size upfront while avoiding the time-consuming iterative search process.
Solution Approach 2:
The patent extracts and removes redundant parameters from the model at initialization using structured pruning, directly reducing model size without requiring the extensive search and training cycles that NAS methods demand. This extraction happens once rather than through repeated search-train-evaluate loops.
Data Source
AI summary
A method, system, and transitory computer-readable media for providing rapid deployment of Deep Neural Networks (DNNs) for Edge Computing using Structured Pruning at Initialization (SPaI). An input of a dense model and a pruning amount is received. Unstructured Pruning (UP) of the input model is performed by the pruning amount to generate a sparse model pruned by the pruning amount. The sensitivity of each layer of the sparse model to pruning is evaluated by contrasting a sparsity of the sparse model with an average sparsity of a global model to generate a structured pruning plan. The structured pruning plan is applied to the resilient layers. A remaining sensitive layers are reinitialized to produce an initialized model pruned by the pruning amount. The initialized model pruned by the pruning amount is provided as an output.