Processor Dot Product Acceleration with Reduced Bit Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face challenges in reducing precision from 32-bit floating-point numbers (FP32) while maintaining accuracy, which hinders operation speed and power consumption optimization.
Innovation Solution
A method of operating a processor that involves arranging weights and activations with fewer bits than the minimum operation unit in input registers, performing multiplications, and outputting the result as a dot product in a preset bit range, optimizing the arrangement and multiplication process to accelerate computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the precision of neural network operations is reduced from FP32 to fewer bits, then operation speed increases and power consumption decreases, but the accuracy of the neural network deteriorates
Solution Approach 1:
The patent segments the dot product computation into multiple stages: first computing partial products of weight and activation values with reduced bit precision, then accumulating these partial products to obtain the final result. This segmentation allows the system to use lower precision arithmetic operations while maintaining overall computational accuracy through the accumulation process.
Solution Approach 2:
The patent changes the precision parameter of the computation from standard FP32 to reduced bit precision formats. By adjusting the number of bits used for representing weight and activation values, the system can trade off between computation speed/ power consumption and accuracy, with the accumulation step compensating for the reduced precision.
2Use of energy by stationary object
If the bit precision is reduced below the minimum operation unit of the processor operator, then power consumption decreases, but the complexity of data arrangement and processing increases
Solution Approach 1:
The patent applies preliminary action by pre-arranging weight and activation values in specific input registers before the dot product computation. The values are organized in a predetermined pattern that facilitates efficient processing by the processor operator, reducing the complexity that would otherwise arise from handling reduced-bit precision data.
Solution Approach 2:
The patent introduces intermediary accumulation registers that hold partial products during the computation process. These intermediary structures mediate between the reduced-bit input values and the final result, simplifying the overall processing by providing a structured way to accumulate partial results without requiring complex real-time processing logic.
3Productivity
If weights and activations are arranged in input registers with specific ordering, then the dot product computation efficiency increases, but the complexity of data arrangement increases
Solution Approach 1:
The patent applies asymmetry by arranging weight and activation values in different ordering patterns in their respective input registers. Rather than using identical arrangements, the asymmetric arrangement optimizes the multiplication and accumulation process, allowing efficient dot product computation while managing the arrangement complexity through systematic patterns.
Data Source
AI summary
A processor, a method of operating the processor, and an electronic device including the processor are disclosed. The method includes arranging, in respective input registers, weights and activations having a smaller number of bits than a minimum operation unit of an operator included in the processor, performing a multiplication between values stored in the input registers, storing a result of the multiplication in an output register, and outputting, from the output register, a value in a preset bit range as a result of a dot product between a first vector including the weights and a second vector including the activations.