Low-bitwidth Neural Network Quantization via Sign-Directed Lookup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep convolutional neural networks (DCNs) face efficiency and accuracy issues when deployed on embedded IoT devices with limited resources due to their high computational requirements and memory constraints, particularly in few-bitwidth neural networks where conventional quantization and arithmetic operations become inefficient.
Innovation Solution
The method involves converting activations to non-negative values and selecting weight values based on the sign of the activations for multiplication, followed by quantization, which reduces weight memory consumption while maintaining precision and improving model accuracy through sign-directed quantization and weighting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional quantization and arithmetic operations are used in few-bitwidth neural networks, then memory consumption is reduced, but computational efficiency and accuracy deteriorate
Solution Approach 1:
The patent segments the weight values into two parts: a magnitude component and a sign component. The sign component is extracted and used to selectively activate pre-computed lookup table entries, while the magnitude component indexes the lookup table. This segmentation allows the system to use simplified arithmetic operations (bitwise AND, lookup, conditional addition) instead of full multiplication, improving computational efficiency while maintaining low memory consumption.
Solution Approach 2:
The patent pre-computes and stores weight magnitudes in lookup tables during an offline phase. During inference, these pre-computed values are retrieved using the activation value as an index, eliminating the need for real-time multiplication operations. This preliminary action significantly speeds up computation while the compact storage format keeps memory usage low.
2Quantity of substance
If low-bitwidth quantization is applied to reduce memory usage, then model accuracy deteriorates
Solution Approach 1:
The patent changes the representation parameters of weights by separating magnitude and sign, and by using sign-directed selective activation. This parameter transformation allows the system to maintain effective precision through the interaction of quantized magnitudes from lookup tables with sign-based selection, achieving accurate results even with low-bitwidth activations and weights.
Solution Approach 2:
The patent introduces sign bits as an intermediary mechanism that mediates between the quantized weight magnitudes and the final computation result. The sign bits selectively activate or deactivate pre-computed lookup table entries, enabling the system to recover precision information that would otherwise be lost in low-bitwidth quantization, thus maintaining model accuracy.
3Adaptability or versatility
If DCNs are deployed on embedded IoT devices, then application scope is expanded, but resource constraints are exceeded
Solution Approach 1:
The patent employs lookup tables with pre-computed weight magnitudes that can be efficiently stored in compact memory. These lookup tables serve as disposable pre-computed structures that enable accurate low-bitwidth inference without requiring large amounts of on-chip memory, making DCNs viable for resource-constrained IoT devices while expanding application scope.
Data Source
AI summary
A method for operating a low-bitwidth neural network includes converting a first activation to a non-negative value (e.g., absolute value). The first activation has a signed value. The sign of the activation is used to select a weight value. A product of the non-negative activation and the selected weight value is computed to determine a next activation. The next activation is quantized and supplied to a subsequent layer of the low-bitwidth neural network.