Neural Network Weight Compression With Blockwise Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large neural networks require significant storage and computational resources, posing challenges for deployment on devices with limited processing power and memory, such as mobile devices, due to their large size and complexity.
Innovation Solution
The development of techniques for neural network model compression and decompression, including quantization, partitioning, and syntax elements, which allow for the encoding and decoding of neural network models to reduce storage and computational requirements, enabling efficient deployment on resource-constrained devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural network models are deployed on devices with limited resources, then model performance can be achieved, but the large size and complexity of the models consume excessive storage and computational resources
Solution Approach 1:
The patent divides the neural network model into multiple blocks, where each block contains a subset of weight coefficients. This segmentation allows for selective processing and storage of model parameters, reducing the computational burden on resource-constrained devices while maintaining overall model performance.
Solution Approach 2:
The patent applies quantization to transform full-precision weight coefficients into lower-precision representations. This parameter change reduces the storage requirements and computational complexity of the neural network model, enabling deployment on devices with limited resources while preserving acceptable model performance.
2Volume of stationary object
If neural network models are compressed to reduce size, then storage requirements are reduced, but the complexity of compression and decompression processes increases
Solution Approach 1:
The patent performs quantization and block partitioning during the model compression phase, preparing the neural network in advance for efficient storage and deployment. This preliminary action reduces the computational complexity during runtime decompression, as the model structure is already optimized for resource-constrained environments.
Solution Approach 2:
By organizing weight coefficients into structured blocks with associated syntax elements, the patent creates a systematic compression framework. This segmentation simplifies the decompression process, as the block structure provides clear organization for selective reconstruction of model parameters.
3Device complexity
If quantization is applied to reduce precision of weight coefficients, then storage and computational requirements are reduced, but the precision of model parameters deteriorates
Solution Approach 1:
The patent applies different quantization strategies to different blocks of the neural network model based on their specific characteristics. This local quality approach allows critical blocks to maintain higher precision while less critical blocks use lower precision, optimizing the balance between model accuracy and resource efficiency.
Solution Approach 2:
The patent systematically transforms weight coefficients from full-precision floating-point representations to lower-precision quantized formats. This parameter change is performed with controlled precision loss, maintaining acceptable model performance while significantly reducing storage and computational requirements for deployment on resource-constrained devices.
Data Source
AI summary
Aspects of the disclosure provide methods and apparatuses for neural network model compression/decompression. In some examples, an apparatus for neural network model decompression includes receiving circuitry and processing circuitry. The processing circuitry decodes, from a bitstream corresponding to a representation of a neural network, at least a syntax element to be applied to multiple blocks in the neural network. Then, the processing circuitry reconstructs, from the bitstream, weight coefficients in the blocks based on the syntax element.