RNN Accelerator Core Weight Reuse via Segmented Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional recurrent neural networks (RNNs) face challenges with weight reusage, leading to high energy consumption and latency due to the serial processing of inputs, which results in repeated loading of weights from memory at each time step.
Innovation Solution
The implementation of RNN accelerator cores that employ data mapping methods to maximize weight reusage by loading weights only once from memory, using a partitioned architecture that separates input and hidden vector weight matrices for efficient matrix multiplication and activation processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If serial processing is used in conventional RNNs, then the network can process inputs progressively at each time step, but weights must be reloaded from memory at each time step resulting in high energy consumption
Solution Approach 1:
The patent segments the RNN computation into distinct phases: a precomputation phase where weight matrices are loaded once and stored in on-chip memory, and an execution phase where inputs are processed progressively using the preloaded weights. This segmentation allows the system to maintain serial processing capability while eliminating repeated weight reloading, thereby resolving the contradiction between progressive processing and energy consumption
Solution Approach 2:
The patent applies preliminary action by precomputing and storing weight matrices in on-chip memory before the actual RNN execution. This preliminary loading of weights eliminates the need for repeated memory access during time step processing, allowing progressive input processing to continue while significantly reducing energy consumption associated with weight reloading
2Reliability
If weights are reloaded from memory at each time step, then the RNN can maintain accurate computations, but the latency increases due to repeated memory access
Solution Approach 1:
The patent segments memory access operations from the time step processing operations. Weight matrices are loaded once into on-chip memory during a precomputation phase, then remain available for all subsequent time steps without requiring repeated external memory access. This segmentation maintains computation accuracy while eliminating latency caused by repeated weight reloading
Solution Approach 2:
The patent performs preliminary loading of weight matrices into on-chip memory before RNN execution begins. This preliminary action ensures that all weights are available locally during processing, maintaining computational accuracy while avoiding the latency penalties of repeated external memory access during time step operations
3Adaptability or versatility
If conventional RNN architecture is used, then the network can process sequential data, but the same weights are repeatedly loaded from memory leading to inefficient weight reusage
Solution Approach 1:
The patent segments the weight management process into a one-time loading phase and a reuse phase. Weight matrices are loaded once into on-chip memory and then reused across all time steps and multiple RNN instances. This segmentation maintains the ability to process sequential data while dramatically improving weight reusage efficiency by eliminating repeated loading operations
Solution Approach 2:
The patent makes the loaded weight matrices universal by having them serve multiple RNN instances and multiple time steps simultaneously. The on-chip memory stores weight matrices that can be shared across different RNN computations, allowing the same weights to be reused efficiently while maintaining the sequential data processing capability of conventional RNNs
Data Source
AI summary
Systems and methods are provided for a neural network that includes a multiply accumulate (MAC) unit that is configured to receive an input vector weight matrix; multiply the input matrix by the input vector weight matrix, generating input vector partial sums; receive time-delayed hidden vectors and a hidden vector weight matrix; and multiply the time-delayed hidden vectors and the hidden vector weight matrix, which generates hidden vector partial sums. An accumulator may be coupled to the MAC unit and configured to accumulate and add the input vector partial sums and the hidden vector partial sums, generating full sum vectors. The neural network may generate the time-delayed hidden vectors based on the full sum vectors. The neural network may further include a first selection device coupled to the MAC unit that is configured to select between the input matrix and the time-delayed hidden vectors for reception at the MAC unit.