MAC Calculation Unit With Quantized Exponential Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network implementations face inefficiencies in execution speed, memory usage, and accuracy due to suboptimal weight representation and lack of efficient hardware solutions, particularly in resource-constrained environments.
Innovation Solution
A calculation unit that employs a floating-point format with a quantized exponential representation for weights, utilizing a RISC processor architecture to perform multiplication and accumulation operations efficiently, minimizing hardware requirements and optimizing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If 8-bit fixed-point formats or 32-bit floating-point format are used for weights, then compatibility with general purpose processors is improved, but memory usage increases and execution speed decreases
Solution Approach 1:
The weight representation is segmented into two parts: a sign bit and a 3-bit exponent field. This segmentation allows the weight to be represented as w = s × 2^e, where s is the sign and e is the exponent. This segmented representation enables efficient hardware implementation where multiplication becomes addition of exponents, significantly improving execution speed while maintaining compatibility with standard processor architectures.
Solution Approach 2:
The patent changes the parameter representation from traditional fixed-point or full floating-point formats to a customized exponential format with a 3-bit exponent field. This parameter change allows the system to achieve faster execution by transforming multiplication operations into simpler exponent addition operations, while still maintaining compatibility with general-purpose processors through the standardized sign-exponent structure.
2Quantity of substance
If quantized neural networks with 8-bit fixed-point formats are used, then memory usage is reduced, but accuracy of the inferential process deteriorates
Solution Approach 1:
The patent introduces a sign-exponent representation where weights are expressed as w = s × 2^e with a 3-bit exponent. This parameter change allows for a wider dynamic range compared to traditional 8-bit fixed-point formats, enabling both very small and very large weights to be represented accurately. The exponential representation maintains memory efficiency (still using 8 bits per weight) while significantly improving accuracy by avoiding the precision loss inherent in fixed-point quantization.
3Productivity
If dedicated hardware accelerators are used, then execution speed is improved, but device complexity and cost increase
Solution Approach 1:
The patent replaces complex hardware multiplication circuits with simpler exponent addition circuits. By representing weights in exponential form (w = s × 2^e), the multiplication operation w × x transforms into adding the exponents: 2^(e1 + e2). This substitution of mechanical multiplication hardware with arithmetic addition logic significantly reduces device complexity and hardware requirements while maintaining fast execution speed.
Data Source
Figure 1~3
Figure 4
Figure 5
AI summary
A calculation unit includes: a multiplier (16), configured to calculate a product (wijxj) of a first factor (wij) and a second factor (xj); an accumulation memory element (18), containing a current accumulation value (ACCj); and a floating-point sum/subtraction unit (17), coupled to the multiplier (16; 116) and the accumulation memory element (18) to receive respectively the product (wijxj) and the current accumulation value (ACCj) and configured to calculate an updated accumulation value (ACCj+1) based on the sum of the product (wijxj) and the current accumulation value (ACCj) and to store the updated accumulation value (ACCj+1) in the accumulation memory element (18). The first factor (wij) and the second factor (xj) each comprise a respective sign bit (SGN(wij), SGN(xj)) and respective exponent bits (EXP(wij)).