Bit-Parallel Vector Composability for Neural Accelerator Energy Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network accelerators face challenges in achieving energy efficiency and performance during vector dot-product operations, which are crucial for ANN training and inference tasks.
Innovation Solution
The proposed solution involves a hardware accelerator architecture that utilizes bit-parallel vector composability, where each vector computation engine processes a subset of bits, and the outputs are aggregated to perform energy-efficient dot-product operations. This design intersperses bit parallelism with data-level parallelism, dynamically composing Narrow-Bitwidth Vector Engines to reduce energy and area costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If conventional neural network accelerators perform vector dot-product operations, then computation is achieved, but energy consumption and area costs are high
Solution Approach 1:
The patent divides the vector dot-product computation into bit-level segments, where each vector computation engine processes a specific subset of bits (e.g., 2-bit slices) independently. This segmentation allows parallel processing of multiple bit-slices across different engines, reducing the energy cost per operation while maintaining overall computation throughput through coordinated aggregation of partial results
2Area of stationary object
If vector computation engines process full-bitwidth vectors, then computation accuracy is maintained, but area costs increase
Solution Approach 1:
Instead of using single wide-bitwidth computation units, the patent segments the bitwidth processing across multiple narrow-bitwidth vector computation engines. Each engine handles a specific bit-slice (e.g., 2 bits) with reduced area footprint, while the collection of engines collectively processes the full-bitwidth vector through parallel operations, achieving both area efficiency and precision preservation
Solution Approach 2:
The patent merges the outputs from multiple narrow-bitwidth vector computation engines through aggregation logic to reconstruct the full-precision dot-product result. This combining approach allows the system to achieve full-bitwidth computation accuracy using only narrow-bitwidth engines, thereby reducing the area cost while maintaining precision
3Use of energy by moving object
If bit-parallel vector composability is implemented, then energy efficiency improves, but device complexity increases
Solution Approach 1:
The patent implements dynamic composition of vector computation engines where the number and configuration of active engines can be adjusted based on the specific computation requirements. This dynamic architecture allows the system to optimize energy efficiency by activating only the necessary number of engines for each operation, while managing complexity through programmable control logic that adapts to different computational tasks
Data Source
AI summary
Methods, apparatus and systems that relate to hardware accelerators of artificial neural network (ANN) performance that significantly reduce the energy and area costs associated with performing vector dot-product operations in the ANN training and inference tasks. Specifically, the methods, apparatus and systems reduce the cost of bit-level flexibility stemming from aggregation logic by amortizing related costs across vector elements and reducing complexity of the cooperating narrower bitwidth units.