Asymmetric Quantization Compilation with Combined Bias
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models face challenges in balancing computational accuracy and hardware design concerns due to asymmetric quantization, which increases circuit critical path, latency, power consumption, and hardware cost.
Innovation Solution
A compiler converts asymmetric operations in neural network models into symmetric operations by merging zero points and scaling factors into combined bias values, reducing hardware requirements and bit width.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If asymmetric quantization is used to fully utilize the quantized fixed-point range and reduce quantization error, then computational accuracy is improved, but hardware complexity increases due to additional zero point hardware logic
Solution Approach 1:
The patent merges the zero point adjustment operation with the existing multiplication operation in the neural network layer. By combining these two operations into a single multiply-accumulate unit, the hardware complexity is reduced while maintaining the ability to perform asymmetric quantization. The merged operation eliminates the need for separate zero point addition hardware logic.
Solution Approach 2:
The patent creates a universal neural network layer that can handle both symmetric and asymmetric quantization cases through a single hardware design. The layer uses a unified multiply-accumulate operation that can accommodate different quantization schemes without requiring separate hardware paths, thereby reducing overall hardware complexity while maintaining computational accuracy.
2Measurement precision
If asymmetric quantization is used to map floating-point range to fixed-point range, then quantization error is reduced, but circuit critical path increases and maximum frequency is limited
Solution Approach 1:
The patent combines multiple operations into a single critical path operation, reducing the number of sequential steps. By merging zero point adjustment with multiplication, the circuit critical path is shortened, allowing higher operating frequencies while maintaining the accuracy benefits of asymmetric quantization.
3Measurement precision
If asymmetric quantization is used to fully utilize quantized bits, then computational accuracy is improved, but power consumption increases
Solution Approach 1:
The patent reduces power consumption by merging operations that would otherwise require separate hardware units. The unified multiply-accumulate operation performs multiple functions simultaneously, reducing the overall hardware area and power consumption while maintaining the computational accuracy needed for fully utilizing quantized bits.
4Measurement precision
If asymmetric quantization is used to map floating-point range to fixed-point range, then quantization error is reduced, but hardware cost increases
Solution Approach 1:
The patent reduces hardware cost by combining multiple hardware components into a single integrated unit. The merged multiply-accumulate operation eliminates the need for separate zero point adjustment hardware, reducing chip area and manufacturing cost while maintaining the quantization accuracy benefits.
Solution Approach 2:
The patent designs a universal neural network layer that can handle both symmetric and asymmetric quantization with the same hardware architecture. This multi-functionality eliminates the need for separate hardware implementations, reducing overall hardware cost while maintaining quantization accuracy.
Data Source
AI summary
A system receives a neural network model that includes asymmetric operations. Each asymmetric operation includes one or more fixed-point operands that are asymmetrically-quantized from corresponding floating-point operands. The system compiles a given asymmetric operation of the neural network model into a symmetric operation that includes a combined bias value. A compiler computes the combined bias value is a constant by merging at least zero points of input and output of the given asymmetric operation. The system then generates a symmetric neural network model including the symmetric operation for inference hardware to execute in fixed-point arithmetic.