Input Aligner for Neural Network Memory Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks (ANNs) face a bottleneck in memory read bandwidth, particularly in neural network architectures that require a large number of MAC units, leading to inefficient hardware processing and high power consumption.
Innovation Solution
The implementation of an input aligner in the NN processor, which uses multibit registers and circuits to load, shift, and reuse data from memory, reducing the need for repeated memory reads and optimizing memory bandwidth by distributing input data or weights to processing elements in a single cycle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a large number of MAC units are used in neural network architecture, then computational capability is improved, but memory read bandwidth requirements increase
Solution Approach 1:
The input aligner pre-processes input data by loading it into registers and performing alignment operations before the main computational task. This preliminary action ensures that data is ready in the correct format and location for efficient processing, reducing the need for repeated memory reads during computation.
Solution Approach 2:
The input aligner acts as an intermediary component between memory and the MAC units. It receives raw input data from memory, performs alignment and formatting operations, and delivers prepared data to the processing elements. This intermediary function reduces the memory bandwidth burden by minimizing direct memory accesses.
2Reliability
If repeated memory reads are performed, then data availability is improved, but memory bandwidth consumption increases
Solution Approach 1:
The input aligner creates a copy of the input data in register storage after the initial memory read. This copy can then be reused multiple times for different computational operations, eliminating the need for repeated memory reads and reducing bandwidth consumption while maintaining data availability.
Solution Approach 2:
The system recovers and reuses input data that would otherwise be discarded or re-read. By storing aligned data in registers and leveraging it across multiple computational steps, the system recovers the value of the initial memory read and eliminates redundant reads, reducing overall bandwidth consumption.
3Speed
If data is loaded and aligned efficiently, then processing speed is improved, but device complexity increases
Solution Approach 1:
The input aligner is implemented as a separate, modular unit that handles alignment operations independently from the main computational architecture. This segmentation allows the alignment function to be optimized without complicating the overall system, as the aligner can be implemented using standard register and memory operations.
Data Source
AI summary
A novel and useful system and method of input alignment for streamlining vector operations that reduce the required memory read bandwidth. The input aligner as deployed in the NN processor, functions to facilitate the reuse of data read from memory and to avoid having to re-read that data in the context of neural network calculations. The input aligner functions to distribute input data (or weights) to the appropriate compute elements while consuming input data in a single cycle. Thus, the input aligner is operative to lower the required read bandwidth of layer input in an ANN. This reflects the fact that normally in practice, a vector multiplication is performed every time instance. This considers the fact that in many native calculations that take place in an ANN, the same data point is involved in multiple calculations.