Decimal-Bit Network Quantization for CNN Model Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 1-bit CNN models, while achieving significant compression and speed-up, lack a solution for further compression and speed-up without compromising model accuracy.
Innovation Solution
The Decimal-Bit Network Quantization (DebNeQ) method allocates a 1-bit convolutional kernel subset to CNN convolutional layers, reducing the number of bits required to represent weights from K×K to N bits, leading to a compressed and accelerated CNN model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If 1-bit convolutional kernels are used in CNN models, then model compression and computational speed-up are achieved, but model accuracy deteriorates
Solution Approach 1:
The patent changes the bit-depth parameter of convolutional kernels from extreme 1-bit quantization to decimal-bit quantization (2-10 bits), achieving an optimal balance between compression ratio and model accuracy. This parameter adjustment allows the model to maintain sufficient precision while still achieving significant compression and speed-up compared to full-precision models.
Solution Approach 2:
The patent introduces dynamic element-wise scaling factors that are learned during training and applied during inference. These scaling factors dynamically adjust the quantized weights to compensate for information loss, enabling the model to maintain high accuracy even with reduced bit-depth representation.
2Quantity of substance
If 1-bit convolutional kernels are used in CNN models, then storage requirements are reduced, but model accuracy deteriorates
Solution Approach 1:
The patent optimizes the bit-depth parameter to decimal values (2-10 bits) rather than using extreme 1-bit quantization, achieving an optimal trade-off between storage compression and model accuracy. This allows significant reduction in storage requirements while preserving sufficient information for accurate inference.
Solution Approach 2:
The patent introduces scaling factors as intermediary parameters that mediate between the quantized weights and the final computation. These scaling factors compensate for the information loss from quantization, allowing the model to maintain high accuracy even with reduced storage requirements.
3Manufacturing precision
If full-precision convolutional kernels are used in CNN models, then model accuracy is maintained, but computational complexity and processing time increase
Solution Approach 1:
The patent reduces the precision parameter from full-precision (32-bit) to decimal-bit (2-10 bit) quantization, significantly simplifying computational operations while maintaining acceptable accuracy through the use of learned scaling factors and optimized quantization schemes.
Solution Approach 2:
The patent replaces complex floating-point arithmetic operations with simpler integer arithmetic operations after quantization. This substitution of computational mechanics dramatically reduces processing complexity and accelerates inference while maintaining model accuracy through carefully designed quantization and scaling mechanisms.
Data Source
AI summary
The disclosure relates to decimal-bit network quantization of CNN models. Methods, apparatus, systems, and articles of manufacture for quantizing a CNN model includes, for a convolutional layer of the CNN model: allocating a 1-bit convolutional kernel subset to the convolutional layer, wherein the convolutional layer includes 32-bit or 16-bit floating-point convolutional kernels with a size of K×K and the 1-bit convolutional kernel subset includes 2N 1-bit convolutional kernel candidates with the size of K×K, 1≤N<K×K and both K and N being positive integers; and performing weights quantization of the convolutional layer by selecting 1-bit convolutional kernel candidates from the 1-bit convolutional kernel subset as 1-bit convolutional kernels of the convolutional layer