Booth Recoded Multi-Mode Inner Product Circuit for Neural Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network processing technologies face inefficiencies in low-precision inner-product computations, particularly in massively parallel neural inference engines, which hinder the performance of neural inference chips in computing neural activations efficiently.
Innovation Solution
The implementation of Booth recoding and carry-save adders in neural inference chips to perform multi-mode low-precision inner-product computations, where input and weight tensors are Booth recoded, multiplied, and partial sums are generated and summed to compute neural activations, allowing for efficient computation across various precision levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional multiplication and addition circuits are used for low-precision inner-product computations, then the circuit design is simple, but the computation efficiency is low and accuracy is insufficient
Solution Approach 1:
The patent segments the inner-product computation into multiple precision modes (e.g., 2-bit, 4-bit, 8-bit) with dedicated computation paths for each mode. Each precision mode has its own set of multipliers and adders configured for that specific precision, allowing parallel processing at different precision levels simultaneously, thereby improving overall computation efficiency while managing circuit complexity through modular design
Solution Approach 2:
The patent employs dynamic precision selection where the computation precision can be adjusted based on the specific neural network layer requirements. The circuit can switch between different precision modes dynamically, using higher precision for layers requiring accuracy and lower precision for layers tolerant of approximation, thus optimizing the balance between computation efficiency and accuracy for different workloads
2Measurement precision
If high-precision computations are used, then computation accuracy is improved, but computation speed decreases and energy consumption increases
Solution Approach 1:
The patent changes the precision parameter dynamically based on the specific computation requirements of different neural network layers. By adjusting the precision parameter (e.g., selecting 2-bit, 4-bit, or 8-bit precision), the system can optimize for either speed or accuracy depending on the workload characteristics, achieving high computation speed when precision can be reduced and maintaining high accuracy when needed
3Adaptability or versatility
If multiple precision modes are supported, then adaptability is improved, but device complexity increases
Solution Approach 1:
The patent designs a universal inner-product computation circuit that can operate in multiple precision modes using the same hardware resources. The circuit employs configurable multipliers and adders that can be dynamically configured to work at different precision levels, allowing a single circuit architecture to serve multiple precision requirements without requiring separate dedicated circuits for each precision mode, thus improving adaptability while controlling complexity
Data Source
AI summary
Neural inference chips for computing neural activations are provided. In various embodiments, the neural inference chip is adapted to: receive an input activation tensor comprising a plurality of input activations; receive a weight tensor comprising a plurality of weights; Booth recode each of the plurality of weights into a plurality of Booth-coded weights, each Booth coded value having an order; multiply the input activation tensor by the Booth coded weights, yielding a plurality of results for each input activation, each of the plurality of results corresponding to the orders of the Booth-coded weights; for each order of the Booth-coded weights, sum the corresponding results, yielding a plurality of partial sums, one for each order; and compute a neural activation from a sum of the plurality of partial sums.