Fused Vector Multiplier Dynamic Bit Width Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing units, such as CPUs, GPUs, FPGAs, and ASICs, face challenges in efficiently processing neural network computations due to inflexible bit width configurations and high energy consumption, which limits their ability to handle diverse data specifications and large-scale neural network operations effectively.
Innovation Solution
A processing device with a dynamically configurable computation bit width, featuring a memory, computational circuit, and control circuit that selects appropriate multiplier and adder circuits based on input data and control instructions, utilizing a fused vector multiplier to perform computations with varying bit widths, thereby optimizing performance and power usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If CPU or GPU is used for neural network computations, then the structure is versatile, but computation speed and efficiency are insufficient
Solution Approach 1:
The patent implements dynamic bit width configuration where the computational circuit can adjust the number of bits used for computation based on the specific neural network layer and data requirements. This dynamic adaptation allows the system to optimize between precision and speed for different computation scenarios, resolving the contradiction between versatile structure and computation speed by making the computational parameters flexible rather than fixed.
Solution Approach 2:
The system changes computational parameters (bit width) to optimize performance. By varying the bit width according to the importance and precision requirements of different neural network layers, the system achieves both versatility (handling different precision requirements) and high computation speed (using lower bit widths where precision can be reduced).
2Productivity
If bit width of data is decreased to promote computation efficiency, then computation efficiency improves, but precision loss occurs
Solution Approach 1:
The patent applies different bit widths to different parts of the neural network computation based on local requirements. Important layers that require high precision use larger bit widths, while less critical layers use smaller bit widths. This local differentiation resolves the contradiction by ensuring precision is maintained only where necessary, thereby improving overall computation efficiency without unacceptable precision loss.
Solution Approach 2:
The system dynamically adjusts bit width based on the specific computation task and layer importance. This dynamic configuration allows the system to optimize the precision-efficiency tradeoff for each neural network layer individually, rather than using a fixed bit width for all computations.
3Adaptability or versatility
If FPGA is used for configurable programming on neural networks, then dynamic configurability is achieved, but dominant frequency is low and energy consumption is high
Solution Approach 1:
The patent segments the computational circuit into modular components (multiplier circuits, adder circuits) that can be selectively activated based on the computation requirements. This segmentation allows the system to achieve configurability similar to FPGA but with lower energy consumption by only activating the necessary computational units rather than reconfiguring the entire system.
Solution Approach 2:
The computational circuit is designed with multi-functional units that can perform different operations (multiplication, addition) with variable bit widths using the same hardware resources. This universality provides dynamic configurability without the high energy overhead of FPGA reconfiguration, as the same physical circuit can adapt its behavior through control signals.
4Productivity
If ASIC is used for neural network acceleration, then computation speed improves, but storage device and computation device constraints limit flexibility
Solution Approach 1:
The patent introduces dynamic bit width configuration capability into the ASIC-like fixed architecture. By allowing the bit width to be dynamically adjusted based on the neural network layer requirements, the system maintains high computation speed (characteristic of ASIC) while gaining flexibility to adapt to different data specifications and precision requirements.
5Device complexity
If fixed length data storage and computation is used, then device simplicity is maintained, but ability to support different bit widths is lost
Solution Approach 1:
The system adds dynamic configurability to the data length, allowing the computational circuit to adapt to different bit widths without requiring multiple fixed-length storage units. This dynamic approach maintains relative device simplicity while achieving versatility in supporting different data specifications.
Data Source
AI summary
The present disclosure relates to a fused vector multiplier for computing an inner product between vectors, where vectors to be computed are a multiplier number vector {right arrow over (A)}{AN . . . A2A1A0} and a multiplicand number {right arrow over (B)} {BN . . . B2B1B0}, {right arrow over (A)} and {right arrow over (B)} have the same dimension which is N+1. The multiplier includes: N+1 multiplication sub-units configured to perform multiplication on each dimension of a vector respectively, and take lower n bits of the multiplier number vector for multiplication each time, where the n bits are removed from the binary number of each dimension of the multiplier number vector after the n bits are taken, and n is larger than 1 and less than N+1; an adder tree configured to perform addition on results of N+1 multiplication sub-units obtained from a same operation each time; and a result register configured to hold a result of every addition performed by the adder tree and send the result to the adder tree for next computation.