Neural Network Processing Unit With Approximate Multiplier
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing units face challenges in efficiently processing complex input data while maintaining accuracy, particularly in low-power and high-performance embedded systems like smartphones.
Innovation Solution
The implementation of neural network processing units that incorporate an approximate multiplier and perform stochastic rounding operations, enabling reduced computational requirements while maintaining accuracy through fixed-point type training computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional neural network processing units perform exact multiplication operations to maintain accuracy, then computation precision is improved, but computational overhead and power consumption increase
Solution Approach 1:
The patent changes the parameter of multiplication operation from exact to approximate by introducing a rounding mechanism. The approximate multiplier performs multiplication followed by rounding to a specified precision, transforming the operation from high-precision to controlled-precision computation. This parameter change reduces computational complexity while maintaining sufficient accuracy for neural network operations.
Solution Approach 2:
The patent employs approximate computation that sacrifices exactness for efficiency. The approximate multiplier uses simplified computation logic that is computationally 'cheaper' and faster, accepting that the result is an approximation rather than an exact value. This trade-off is acceptable in neural network contexts where approximate results suffice.
2Use of energy by moving object
If neural network processing units reduce computational operations to lower power consumption, then energy efficiency is improved, but accuracy of processing complex input data deteriorates
Solution Approach 1:
The patent introduces stochastic rounding as a parameter change in the computation process. Instead of deterministic rounding, stochastic rounding adds a random component that distributes quantization errors more evenly across computations. This maintains numerical stability and accuracy over multiple operations while using fewer computational resources.
Solution Approach 2:
The patent implements accumulation of rounded values followed by stochastic rounding of the accumulated result. This feedback mechanism where intermediate results are accumulated and then rounded reduces the impact of individual rounding errors, maintaining overall computation accuracy while using efficient approximate multiplication operations.
3Productivity
If approximate multiplication is used to reduce computational requirements, then productivity is improved, but loss of accuracy increases
Solution Approach 1:
The patent uses feedback through accumulation of multiple approximate multiplication results before performing final stochastic rounding. By accumulating intermediate values and only rounding the final accumulated result, the system recovers from individual rounding errors, minimizing overall accuracy loss while maintaining high processing speed through efficient approximate multiplication operations.
Data Source
AI summary
A neural network processing unit may be configured to perform an approximate multiplication operation and a system on chip may include the neural network processing unit. The neural network processing unit may include a plurality of neural processing units and may perform a computation based on one or more instances of input data and a plurality of weights. At least one neural processing unit is configured to receive a first value and a second value and perform an approximate multiplication operation based on the first value and the second value and is further configured to perform a stochastic rounding operation based on an output value of the approximate multiplication operation.