Neural Network Shifter Circuit Using Log-Quantized Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of multiplication operations in neural networks leads to significant power consumption and physical footprint requirements for MAC arrays, making neural network-based algorithms challenging for edge devices due to the need for extensive arrays to process data in a timely manner.
Innovation Solution
The introduction of a second-generation neural network processor (NN 2.0) that utilizes shifter functions instead of multipliers and adders, log-quantizing parameters to allow for shift operations, reducing power consumption and footprint while accelerating calculations by replacing multiplication and addition operations with shift operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If MAC arrays are used to perform multiplication and addition operations in neural networks, then calculation accuracy is maintained, but power consumption and physical footprint increase significantly
Solution Approach 1:
The patent applies parameter changes by transforming the neural network weights into log-quantized format. Instead of using standard floating-point or integer weights that require complex multiplication operations, the weights are represented as logarithmic values. This allows the multiplication operation to be replaced with simpler addition and bit-shifting operations, significantly reducing power consumption while maintaining computational accuracy. The log-quantized parameters enable the system to perform neural network computations with reduced hardware complexity.
Solution Approach 2:
The patent substitutes the mechanical multiplication and addition operations performed by MAC arrays with a different computational mechanism based on logarithmic properties. By using log-quantized parameters, the system replaces the need for complex arithmetic units with simpler logic circuits that perform addition and bit-shifting. This mechanical substitution reduces the power consumption and physical footprint while maintaining the essential function of neural network computation.
2Measurement precision
If MAC arrays are used to process neural network operations, then computational precision is maintained, but device complexity and physical footprint increase
Solution Approach 1:
The patent transforms the parameter representation from standard numerical formats to log-quantized format. This parameter change fundamentally alters the computational approach, allowing neural network operations to be performed using simpler hardware logic. The log-quantized parameters enable the system to maintain computational precision while reducing the complexity of the hardware required to perform the operations.
Solution Approach 2:
The patent replaces the complex mechanical system of MAC arrays with a simpler computational mechanism based on logarithmic properties. By substituting multiplication and addition operations with addition and bit-shifting operations on log-quantized parameters, the system reduces hardware complexity while maintaining computational accuracy.
3Productivity
If extensive MAC arrays are deployed to process data in a timely manner, then processing speed is improved, but power consumption and footprint increase
Solution Approach 1:
The patent substitutes the power-intensive MAC array operations with a more efficient computational mechanism. By using log-quantized parameters and replacing multiplication with addition and bit-shifting operations, the system achieves comparable processing speed with significantly reduced power consumption. This mechanical substitution allows edge devices to process neural network operations in real-time without the power budget required by traditional MAC array approaches.
Solution Approach 2:
The transformation to log-quantized parameters enables more efficient processing by changing the mathematical representation of neural network weights. This parameter change allows the system to perform computations with fewer power-intensive operations while maintaining processing speed, making real-time neural network inference feasible on power-constrained edge devices.
4Quantity of substance
If traditional quantization is applied to reduce parameter size, then model size is reduced, but multiplication operations still require significant power
Solution Approach 1:
The patent goes beyond traditional quantization by applying logarithmic quantization to the parameters. Instead of simply reducing the bit-width of standard numerical representations, the system transforms parameters into log-quantized format. This fundamental parameter change allows the system to maintain compact model size while enabling power-efficient computation through the replacement of multiplication operations with addition and bit-shifting operations.
Data Source
AI summary
Aspects of the present disclosure involve systems, methods, computer instructions, and artificial intelligence processing elements (AIPEs) involving a shifter circuit or equivalent circuitry/hardware/computer instructions thereof configured to intake shiftable input derived from input data for a neural network operation; intake a shift instruction derived from a corresponding log quantized parameter of a neural network or a constant value; and shift the shiftable input in a left direction or a right direction according to the shift instruction to form shifted output representative of a multiplication of the input data with the corresponding log quantized parameter of the neural network.