Bit-Shifted Neural Network Parameters for Lower-Power Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks consume significant power, and designing dedicated low bit-width hardware is challenging and expensive.
Innovation Solution
Bit shifting neural network parameters by multiplying them by 2^B-b and dividing the quantization scale by 2^B-b to simulate low bit width quantization, ensuring the least significant bits are zero, thereby reducing power consumption without requiring explicit hardware support for low bit width.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If dedicated low bit-width hardware is designed and produced, then power consumption is reduced, but device complexity and manufacturing cost increase
Solution Approach 1:
The patent enables standard high bit-width hardware to process low bit-width neural network parameters by bit-shifting them to align with the hardware's native bit-width. This allows a single hardware design to handle both low bit-width and high bit-width operations without requiring dedicated low bit-width hardware, thereby reducing device complexity while maintaining power consumption benefits
Solution Approach 2:
The patent changes the representation parameters of neural network parameters by bit-shifting them. Low bit-width parameters are shifted left by (B-b) positions to fit into B-bit hardware registers, with the upper bits set to zero. This parameter transformation allows the same hardware to process different bit-width data types efficiently
2Use of energy by moving object
If dedicated low bit-width hardware is designed and produced, then power consumption is reduced, but manufacturing cost increases
Solution Approach 1:
Instead of creating specialized low bit-width hardware, the patent creates a software-based representation layer that copies the functionality of low bit-width processing onto standard hardware. The bit-shifting operation creates a virtual low bit-width environment within the high bit-width hardware, eliminating the need for expensive specialized manufacturing
Solution Approach 2:
Standard hardware processors are made universal by enabling them to handle both low bit-width and high bit-width neural network parameters through bit-shifting. This eliminates the need for separate dedicated hardware designs for different bit-width requirements, reducing manufacturing costs while maintaining energy efficiency
3Use of energy by moving object
If bit shifting is applied to neural network parameters, then power consumption is reduced by increasing zero bits, but computational precision may be affected
Solution Approach 1:
The patent applies partial quantization by bit-shifting parameters to create leading zero bits rather than eliminating precision entirely. By shifting left by (B-b) positions and setting upper bits to zero, the method creates a partial reduction in precision that achieves power savings while maintaining sufficient accuracy for neural network operations
Solution Approach 2:
The patent transforms the precision parameters of neural network parameters through bit-shifting operations. The original precision is adjusted by the shift amount, creating a new precision level that balances power consumption requirements with acceptable computational accuracy for the specific application
Data Source
AI summary
A processor-implemented method includes bit shifting a binary representation of a neural network parameter. The neural network parameter has fewer bits, b, than a number of hardware bits, B, supported by hardware that processes the neural network parameter. The bit shifting effectively multiplies the neural network parameter by 2B-b. The method also includes dividing a quantization scale by 2B-b to obtain an updated quantization scale. The method further includes quantizing the bit shifted binary representation with the updated quantization scale to obtain a value for the neural network parameter.