Neural Network Compression with Joint Pruning and Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern deep neural networks require significant memory and computational resources, making them impractical for deployment on resource-constrained devices such as small form factor, battery-powered devices, and devices with modest memory and processing capabilities, limiting their use in real-time AI applications like intelligent cameras, drones, and augmented/virtual reality systems.
Innovation Solution
A unified sparsification-quantization technique is applied during a single training pass to compress neural networks, reducing their size and computational complexity while maintaining high accuracy, by jointly optimizing for sparse and low-precision weights and activations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a separate GPU and computer vision subsystem are used in parallel, then computational power and processing capability are improved, but device complexity and resource consumption increase
Solution Approach 1:
The patent combines computer vision processing with the graphics pipeline by integrating vision operations into the existing GPU architecture. This allows vision workloads to share the same hardware resources as graphics workloads, eliminating the need for separate vision processing units and reducing overall system complexity while maintaining high computational power
2Adaptability or versatility
If deep neural networks are deployed on resource-constrained devices, then AI functionality is improved, but memory and computational resource requirements cause deployment infeasibility
Solution Approach 1:
The patent applies quantization to change the precision parameters of neural network weights and activations from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integers). This parameter transformation significantly reduces memory requirements and computational resource consumption, enabling deployment on resource-constrained devices while maintaining acceptable AI functionality
3Quantity of substance
If neural network precision is reduced for compression, then device resource requirements are improved, but model accuracy deteriorates
Solution Approach 1:
The patent applies different quantization strategies to different parts of the neural network based on their importance and sensitivity. Critical layers and parameters maintain higher precision while less sensitive components use lower precision, achieving local optimization that balances compression ratio and accuracy retention
4Productivity
If parallel GPU and vision subsystems are used, then processing capability is improved, but energy consumption increases
Solution Approach 1:
The patent designs the GPU subsystem to perform both graphics rendering and computer vision processing functions using the same hardware resources. This multi-functionality allows the system to achieve high processing capability for both workloads while avoiding the additional energy consumption that would result from maintaining separate dedicated vision processing hardware
Data Source
AI summary
A neural network model is trained, where the training includes multiple training iterations. Weights of a particular layer of the neural network are pruned during a forward pass of a particular one of the training iterations. During the same forward pass of the particular training iteration, values of weights of the particular layer are quantized to determine a quantized-sparsified subset of weights for the particular layer. A compressed version of the neural network model is generated from the training based at least in part on the quantized-sparsified subset of weights.