Neural Engine Asymmetric Quantization Bias Precomputation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing systems face inefficiencies due to the computational complexity associated with asymmetric quantization, particularly in convolution operations, which increases power consumption and bandwidth usage when relying solely on central processing units (CPUs).
Innovation Solution
A neural engine circuit is designed to handle asymmetric quantization by adjusting kernel coefficients during runtime and precomputing an adjusted bias for the convolution operation, reducing computational complexity and enhancing efficiency in convolution operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If asymmetric quantization is implemented in neural network processing, then processing accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent precomputes adjustment values for asymmetric quantization during an offline training phase, storing these values in lookup tables. During runtime inference, the neural engine simply retrieves and applies these precomputed values rather than performing complex real-time calculations, thereby maintaining high processing accuracy while significantly reducing computational complexity.
Solution Approach 2:
The patent separates asymmetric quantization processing into two distinct phases: an offline training phase where complex adjustment computations are performed and stored, and an online inference phase where only simple retrieval and application operations are needed. This segmentation allows the system to achieve high accuracy when needed while maintaining low computational complexity during actual neural network execution.
2Adaptability or versatility
If asymmetric quantization processing is performed using CPU, then processing flexibility is maintained, but power consumption increases
Solution Approach 1:
The patent replaces the general-purpose CPU-based asymmetric quantization processing with a specialized neural engine circuit designed specifically for this function. This hardware substitution eliminates the high power consumption associated with CPU execution while maintaining the same processing flexibility and adaptability, as the neural engine can be configured through programmable logic and lookup tables.
Solution Approach 2:
The neural engine circuit performs asymmetric quantization processing autonomously using dedicated hardware components (kernel adjust circuit, multiply-accumulate circuits, accumulator) without requiring CPU intervention. This self-service capability allows the system to maintain processing flexibility through programmable configuration while achieving significant power consumption reduction through specialized hardware execution.
3Adaptability or versatility
If asymmetric quantization is handled through software implementation, then adaptability to different configurations is improved, but processing speed decreases
Solution Approach 1:
The patent replaces software-based asymmetric quantization processing with dedicated hardware circuits including kernel adjust circuits, multiply-accumulate circuits, and accumulators. This hardware implementation executes operations in parallel and at clock-speed rates, achieving processing speeds orders of magnitude faster than software while maintaining configuration adaptability through programmable logic and configurable lookup tables.
Solution Approach 2:
The patent divides asymmetric quantization processing into distinct hardware stages: kernel coefficient adjustment, multiplication, accumulation, and bias application. Each stage is implemented as a separate circuit module that can be independently configured and optimized, enabling both high-speed parallel execution and adaptability to different neural network configurations through programmable control.
4Measurement precision
If precise asymmetric quantization processing is performed, then output accuracy is improved, but bandwidth usage increases
Solution Approach 1:
The patent changes the representation parameters of quantized data by computing and applying adjustment values that account for asymmetric quantization effects. Instead of transmitting or storing multiple precision versions of data, the system uses a single adjustment value parameter applied during processing to achieve high output accuracy, thereby minimizing bandwidth usage while maintaining precision.
Data Source
AI summary
Embodiments relate to performing multiply-accumulator operation on asymmetrically quantized input data and kernel data in a neural processor. Instead of adjusting to the input data at a multiply-accumulator to account for the asymmetric quantization of the input data, an adjusted bias for the multiply-accumulator operation is computed beforehand and stored in the multiply-accumulator. On the other hand, kernel coefficients derived from the kernel data are adjusted at the multiply-accumulator to account for the asymmetric quantization. In this way, computational complexity associated with asymmetric quantization may be reduced while increasing the efficiency of the convolution operations at the neural processor.