MAC-Based Quantization for Smaller Neural Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional electronic devices require a larger multiplier dedicated to quantization, increasing the size of the processor, due to the need for bigger bit operations during quantization processes.
Innovation Solution
An electronic device utilizing a multiplier and an accumulator to perform quantization by obtaining multiplicand data through a convolution operation, storing data based on scale factors, and using a bit shift factor to obtain an integer result without requiring a multiplier dedicated to quantization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a multiplier dedicated to quantization is used to perform big bit operations, then quantization capability is improved, but processor size increases
Solution Approach 1:
The patent combines the quantization multiplication function with the existing MAC unit, allowing the same hardware to perform both convolution multiplication and quantization operations. This merging eliminates the need for a separate dedicated quantization multiplier, thereby maintaining quantization capability while reducing processor area.
Solution Approach 2:
The MAC unit is designed to be multi-functional, serving both as a convolution computation unit and a quantization unit. By making the multiplier universal, it can handle different bit widths and operations (8-bit convolution and 21-bit quantization) without requiring separate dedicated hardware for each function.
2Measurement precision
If a multiplier of bigger bits is used for quantization, then quantization precision is improved, but device complexity increases
Solution Approach 1:
The patent employs dynamic bit shifting and scaling operations to achieve variable precision quantization. Instead of using a fixed large-bit multiplier, the system dynamically adjusts the bit width and precision requirements based on the accumulation results and scale factors, allowing the same hardware to adapt to different precision needs.
Solution Approach 2:
The system changes operational parameters (bit width, scale factors, shift amounts) to achieve different quantization precisions. By modifying these parameters rather than changing the physical multiplier structure, the system maintains high precision while avoiding the complexity of multiple fixed-precision multipliers.
Data Source
AI summary
An electronic device is provided. The electronic device includes memory, a processor, the processor including a multiplier, and an accumulator, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to obtain multiplicand data according to a convolution operation between an activation value and a weight value by using the multiplier and the accumulator and store the data in the memory, based on a first scale factor for quantizing the activation value, a second scale factor for quantizing the weight value, and a third scale factor for quantizing the multiplicand data, obtain multiplier data and a bit shift factor and store them in the memory, and obtain an integer that quantized the multiplicand data according to a multiplication operation between the multiplicand data, the multiplier data and the bit shift factor stored in the memory by using the multiplier.