Multiply-Accumulate Circuit Exponential Weight Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network techniques face challenges in reducing processing load while maintaining quantization granularity, particularly as dimensionality increases, with binary and quaternary quantization methods showing limitations in granularity and accuracy.
Innovation Solution
An information processing device and method that perform multiply-accumulate operations using weight coefficients quantized by exponential representation, where exponents are expressed as fractions with a predetermined divisor, allowing for finer granularity and reduced processing load through optimized multiply-accumulate circuit operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If binary quantization is used to reduce processing load, then computational complexity decreases, but quantization granularity becomes rougher as dimensionality increases
Solution Approach 1:
The patent changes the parameter representation from binary values to exponential form with fractional exponents (n/p where p is a predetermined divisor). This allows the system to maintain fine quantization granularity while reducing computational complexity, as the exponential representation enables efficient calculation through pre-computed lookup tables and reduced arithmetic operations.
Solution Approach 2:
The patent segments the exponent calculation by using a predetermined divisor p to create discrete levels (0, 1/p, 2/p, ..., (p-1)/p). This segmentation allows the system to pre-compute and store values in lookup tables, reducing runtime computational load while maintaining precise quantization granularity through the fractional exponent structure.
2Productivity
If weight coefficients are quantized to reduce computational complexity, then processing speed increases, but accuracy of neural network computations decreases
Solution Approach 1:
The patent transforms weight coefficients into exponential form w = α^(n/p), where the fractional exponent provides fine-grained quantization levels. This parameter transformation enables the system to achieve both high processing speed (through efficient exponential calculation and lookup tables) and high computation accuracy (through fine quantization granularity maintained by the fractional exponent structure).
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
[Object] To reduce a processing load associated with inner product operations while also guaranteeing the quantization granularity of weight coefficients. [Solution] Provided is an information processing device including: a multiply-accumulate operation circuit configured to execute a multiply-accumulate operation on the basis of multiple input values and multiple weight coefficients that are quantized by an exponential representation and that correspond to each of the input values. Exponents of the quantized weight coefficients are expressed by fractions taking a predetermined divisor as a denominator, and the multiply-accumulate operation circuit performs the multiply-accumulate operation using different addition multipliers on the basis of a remainder determined from the divisor.