Neural Network Compression with Unified Pruning and Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern, high-performing deep neural networks require significant memory and computational resources, making them impractical for deployment on resource-constrained devices such as intelligent cameras, drones, and AR/VR systems due to their large memory footprints and computational demands.
Innovation Solution
A unified sparsification-quantization technique is applied during a single training pass to compress neural networks, achieving high compression rates with minimal accuracy loss by simultaneously optimizing for sparse and low-precision weights and activations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a separate GPU and computer vision subsystem are used in parallel, then computational capability is improved, but device complexity and resource consumption increase
Solution Approach 1:
The patent integrates computer vision processing directly into the graphics processing pipeline by implementing convolutional neural network operations within the GPU architecture. This merging eliminates the need for separate parallel subsystems while maintaining high computational capability through unified hardware resources.
2Manufacturing precision
If high-precision neural networks are deployed, then model accuracy is improved, but memory footprint and computational demands increase making deployment impractical on resource-constrained devices
Solution Approach 1:
The patent applies quantization techniques that change the precision parameters of neural network weights and activations from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integers or lower). This parameter transformation significantly reduces memory footprint and computational demands while maintaining acceptable model accuracy through careful quantization strategy and potential fine-tuning.
3Quantity of substance
If neural network compression is applied, then memory footprint is reduced, but model accuracy may deteriorate
Solution Approach 1:
The patent employs selective pruning strategies that identify and remove only the least important connections or neurons in the neural network while preserving critical pathways. This local quality approach ensures that compression is applied non-uniformly across the network, maintaining high accuracy in important regions while achieving overall size reduction. The method may involve importance metrics such as gradient magnitude, activation strength, or sensitivity analysis to guide selective preservation.
Data Source
AI summary
A neural network model is trained, where the training includes multiple training iterations. Weights of a particular layer of the neural network are pruned during a forward pass of a particular one of the training iterations. During the same forward pass of the particular training iteration, values of weights of the particular layer are quantized to determine a quantized-sparsified subset of weights for the particular layer. A compressed version of the neural network model is generated from the training based at least in part on the quantized-sparsified subset of weights.