RNN Acceleration via Parallel Vector Multiplication Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Recurrent neural networks (RNNs) face challenges in accelerating calculations, particularly with large model parameters, leading to increased time consumption and reduced operation efficiency, especially in complex applications like speech recognition and language modeling.
Innovation Solution
A recurrent neural network accelerating system is introduced, featuring a vector multiplication circuit with N groups of multiplication arrays, each including k units, allowing for parallel processing and time-sharing calculations of weight and hidden state data, along with a state updating circuit that optimizes gate operations, thereby improving calculation speed and adaptability to different RNN sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional sequential calculation methods are used for RNN with large model parameters, then calculation accuracy is maintained, but time consumption increases and operation efficiency decreases
Solution Approach 1:
The patent segments the RNN calculation process into multiple independent computation paths. The multiplication arrays are divided into N groups that can operate in parallel, each handling a portion of the weight matrix multiplications. This segmentation allows the system to process large model parameters through distributed parallel computation rather than sequential processing, thereby reducing time consumption while maintaining accuracy.
Solution Approach 2:
The patent introduces a parallelism dimension by implementing N groups of multiplication arrays that operate simultaneously. Instead of processing calculations in a single sequential dimension, the system exploits the parallel dimension to perform multiple weight multiplications at the same time. This dimensional transformation from sequential to parallel processing directly addresses the time efficiency problem without sacrificing computational accuracy.
2Productivity
If parallel processing is implemented to accelerate RNN calculations, then operation efficiency improves, but device complexity increases
Solution Approach 1:
The patent designs the N groups of multiplication arrays to use identical structural configurations and computational logic. Each group serves the same function of performing weight multiplications, but operates on different data segments. This universal design allows the system to achieve parallelism without proportionally increasing device complexity, as the same modular unit is replicated N times rather than creating N different complex components.
Solution Approach 2:
The patent implements dynamic control mechanisms to manage the N parallel multiplication arrays. A control unit dynamically allocates and coordinates the operation of each multiplication array group based on the computational requirements. This dynamic management allows the system to activate only the necessary parallel units for each calculation task, optimizing resource utilization and reducing the effective complexity compared to having all units permanently active.
3Measurement precision
If all weight and hidden state values are combined before accumulation, then calculation accuracy is ensured, but processing speed decreases
Solution Approach 1:
The patent performs preliminary accumulation operations within each multiplication array group before the final combination step. Each group accumulates its partial results independently and preliminarily, then these pre-accumulated results are combined in a final reduction step. This preliminary action allows the system to maintain accuracy through proper accumulation while avoiding the bottleneck of combining all values sequentially before any accumulation occurs, thereby improving processing speed.
Solution Approach 2:
The patent ensures continuous useful action by allowing multiple multiplication array groups to perform their accumulation operations simultaneously and independently. Rather than waiting for all values to be combined before accumulation begins, the system maintains continuous computational flow across all parallel units. This continuity eliminates idle waiting time and keeps the calculation pipeline full, improving processing speed while preserving accuracy through the final combination of all accumulated results.
Data Source
AI summary
A system for accelerating an RNN network including: a first cache, which is used for outputting Wx1 to WxN or Wh1 to WhN in parallel in N paths in a cyclic switching manner, and the degree of parallelism is k; a second cache, which is used for outputting xt or ht-1 in the cyclic switching manner; a vector multiplication circuit, which is used for, by using N groups of multiplication arrays, respectively calculating Wx1xt to WxNxt, or respectively calculating Wh1ht-1 to WhNht-1; an addition circuit, which is used for calculating Wx1xt+Wh1ht-1+b1 to WxNxt+WhNht-1+bN; an activation circuit, which is used for performing an activation operation according to an output of the addition circuit; a state updating circuit, which is used for acquiring ct-1, calculating ct and ht, updating ct-1, and sending ht to the second cache; a bias data cache; a vector cache; and a cell state cache.