LSTM Hardware Accelerator Tile-Based Weight Matrix Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware accelerators for Long Short-Term Memory (LSTM) neural networks face limitations in processing speed due to limited integrated circuit (IC) resources, particularly when dealing with large numbers of neurons, as they struggle to efficiently perform matrix-vector multiplication (MVM) operations, leading to high latency and inefficient use of available resources.
Innovation Solution
The proposed solution involves creating tiles for each weight matrix based on the available on-chip memory, allowing for parallel processing of non-recurrent and recurrent weight matrices with input vectors, optimizing the use of resources and reducing the number of iterations required for MVM operations, thereby accelerating LSTM processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If hardware accelerators use parallel multiply accumulate (MAC) units to increase processing speed, then MVM operation latency is reduced, but the acceleration benefits are limited by available IC resources for large values of m and n
Solution Approach 1:
The weight matrix is divided into multiple smaller matrices called tiles, where each tile can be independently loaded into on-chip memory and processed. This segmentation allows the system to handle large weight matrices by processing them in manageable chunks, reducing the resource requirements for any single operation while maintaining overall processing speed.
2Productivity
If a fixed number of parallel multipliers are used to perform MVM operations with multiple iterations, then the same number of iterations is required regardless of available IC resources, but excess IC resources are not utilized
Solution Approach 1:
The system dynamically adjusts the number of tiles created and processed based on the available on-chip memory resources. When more resources are available, more tiles can be created and processed in parallel, increasing productivity. When resources are limited, fewer tiles are processed, but the system adapts to maintain efficient operation. This dynamic adjustment ensures optimal utilization of available IC resources.
3Productivity
If weight matrices are loaded into on-chip memory for processing, then MVM operations can be performed, but data transfer and memory management become bottlenecks
Solution Approach 1:
The system pre-divides weight matrices into tiles and organizes them in a way that facilitates efficient loading into on-chip memory. By preparing the weight data structure beforehand and optimizing the tile division, the system minimizes data transfer time and memory management overhead during actual processing operations.
Data Source
AI summary
Method for accelerating matrix vector multiplication (MVM), long short-term memory (LSTM) systems and integrated circuits for the same are described herein. In one example, a system for accelerating processing by an LSTM architecture includes a first processing circuitry (FPC) and a second processing circuitry (SPC). The FPC receives weight matrices from a trained neural network, and stores the weight matrices in a first memory circuitry. The SPC stores the weight matrices in a second memory circuitry, and generates an output vector based on the weight matrices and input vectors. The FPC further processes each of the weight matrices for communication to the SPC, and divides each weight matrix into a number of tiles based on an available resources in the second memory circuitry and a size of the weight matrix. The SPC further applies each tile of each weight matrix to a corresponding input vector to generate the output vector.