Neural Network Parameter Bit Shifting for Low-Bit Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks consume significant power, and designing dedicated low bit-width hardware is challenging and expensive, while existing solutions for simulating low bit-width quantization introduce noise and have exponential configuration spaces.
Innovation Solution
Simulate low bit-width quantization by bit-shifting neural network parameters and adjusting quantization scales to ensure the least significant bits are always zero, applicable to existing hardware without explicit support for low bit-width quantization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If neural networks use low bit-width quantization to reduce power consumption, then power consumption decreases, but hardware complexity and manufacturing cost increase due to the need for dedicated low bit-width hardware
Solution Approach 1:
The patent simulates low bit-width quantization effects on standard bit-width hardware by copying the mathematical behavior of low bit-width operations. Instead of using actual low bit-width hardware, the invention uses software-based bit-shifting operations that replicate the power-saving effects of low bit-width quantization while running on conventional hardware, thus avoiding the need for specialized hardware design and manufacturing
Solution Approach 2:
The patent changes the parameter representation by bit-shifting neural network parameters so that least significant bits are always zero. This parameter transformation allows standard hardware to emulate low bit-width quantization behavior, achieving power consumption reduction without requiring actual hardware parameter changes or specialized low bit-width processors
2Use of energy by moving object
If existing solutions for simulating low bit-width quantization are used, then power consumption reduces, but noise is introduced and the configuration space becomes exponential
Solution Approach 1:
The patent applies parameter changes through bit-shifting operations that systematically set least significant bits to zero, creating a structured approach to simulating low bit-width quantization. This method avoids the exponential configuration space of existing solutions by using a simple, deterministic bit-shifting rule rather than complex quantization configurations, thereby reducing noise and improving reliability
Solution Approach 2:
The patent extracts only the essential power-saving mechanism from low bit-width quantization - the effect of having trailing zero bits - and applies it independently through bit-shifting operations. By separating this key benefit from the complex quantization process, the invention avoids introducing the noise and complexity associated with full quantization schemes while still achieving power consumption reduction
Data Source
Figure 1
Figure 2A~2C
Figure 2D
AI summary
A processor-implemented method includes bit shifting a binary representation of a neural network parameter. The neural network parameter has fewer bits, b, than a number of hardware bits, B, supported by hardware that processes the neural network parameter. The bit shifting effectively multiplies the neural network parameter by 2 B-b . The method also includes dividing a quantization scale by 2 B-b to obtain an updated quantization scale. The method further includes quantizing the bit shifted binary representation with the updated quantization scale to obtain a value for the neural network parameter.