Binary Point Fixed Point Arithmetic for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural network (ANN) processors face inefficiencies in performing multiply-accumulate operations and activation functions, leading to suboptimal performance and resource utilization, especially in handling large numbers of connection inputs.
Innovation Solution
The processor incorporates a neural network unit (NNU) with multiple neural processing units (NPUs) that perform multiply-accumulate operations in a pipelined fashion, using a rotater mechanism to efficiently process data and weights, and includes a dynamically configurable architecture to optimize operations for various neural network layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional ANN processors perform multiply-accumulate operations, then computational functionality is achieved, but resource utilization is suboptimal and performance is reduced
Solution Approach 1:
The patent implements a dynamically reconfigurable architecture where the binary point position can be shifted to different locations based on the specific computational requirements of different neural network layers. This dynamic adjustment allows the same hardware resources to be optimized for different precision requirements, improving computational throughput without proportionally increasing the number of multipliers and adders.
Solution Approach 2:
The system changes the parameter of binary point position dynamically. By shifting the binary point to different positions (e.g., from position 0 to position 7), the apparatus can adapt the precision and range of calculations to match the specific needs of different neural network operations, thereby achieving high performance with reduced hardware resources.
2Adaptability or versatility
If fixed precision arithmetic is used, then hardware implementation is simplified, but adaptability to different neural network requirements is reduced
Solution Approach 1:
The apparatus employs dynamic reconfiguration of the binary point position through shift operations. This allows the system to adapt to different precision requirements of various neural network layers (e.g., input layer vs. hidden layers vs. output layer) without requiring separate fixed-precision hardware for each layer, thus improving adaptability while controlling hardware complexity.
3Speed
If more multipliers and adders are used, then computational speed increases, but resource utilization becomes inefficient
Solution Approach 1:
By dynamically changing the binary point position parameter through shift operations, the system can achieve variable precision arithmetic. This allows faster computation for operations that don't require high precision while maintaining accuracy where needed, thereby improving speed without proportionally increasing the number of computational resources and improving overall resource utilization efficiency.
Data Source
Figure 1
Figure 2~3
Figure 4~6A
AI summary
An apparatus includes a plurality of arithmetic logic units each having an accumulator and an integer arithmetic unit that receives and performs integer arithmetic operations on integer inputs and accumulates integer results of a series of the integer arithmetic operations into the accumulator as an integer accumulated value. A register is programmable with an indication of a number of fractional bits of the integer accumulated values and an indication of a number of fractional bits of integer outputs. A first bit width of the accumulator is greater than twice a second bit width of the integer outputs. A plurality of adjustment units scale and saturate the first bit width integer accumulated values to generate the second bit width integer outputs based on the indications of the number of fractional bits of the integer accumulated values and outputs programmed into the register.