RNN Hardware Mapping via Time Multiplexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware designs for recurrent neural networks (RNNs) lack efficient and distributed implementations that can handle varying sizes and characteristics of Legendre Memory Units (LMUs), Projected Long Short Term Memory (LSTM-P) cells, and feed forward cells, leading to suboptimal performance and power consumption in real-time processing applications like automatic speech recognition and keyword spotting.
Innovation Solution
A digital hardware system that employs time multiplexing over a spatially distributed set of computation blocks to process RNNs, with a mapping algorithm that optimizes latency, throughput, and power consumption by partitioning the network and allocating resources iteratively, ensuring compatibility with LMUs, LSTM-P cells, and feed forward cells.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is buffered into memory for processing, then data availability is improved, but latency and power consumption increase
Solution Approach 1:
The patent extracts the RNN processing functionality from traditional memory-based architectures and implements it directly in hardware logic. By taking out the computational operations from sequential memory access patterns and embedding them in dedicated computation blocks, the system eliminates the need to buffer large amounts of data in memory, thereby reducing latency while maintaining data availability for real-time processing
Solution Approach 2:
The patent segments the RNN into discrete computation blocks that can be independently configured and mapped to hardware resources. This segmentation allows the system to process data streams in a pipelined fashion through multiple computation blocks, eliminating the need for large memory buffers while maintaining continuous data flow and reducing end-to-end latency
2Measurement precision
If RNN size and complexity are increased to improve accuracy, then processing performance is improved, but power consumption and hardware size increase
Solution Approach 1:
The patent implements dynamic configuration of computation blocks where the hardware architecture can be reconfigured based on the specific RNN model requirements. Computation blocks can be dynamically enabled or disabled, and resource allocation can be adjusted runtime based on the complexity of the RNN being processed, allowing high accuracy models to be executed only when needed while reducing power consumption during inference with simpler models
Solution Approach 2:
The patent enables parameter changes in the computation blocks to match the specific characteristics of different RNN variants. By dynamically adjusting operational parameters such as precision, activation functions, and connection patterns in the hardware, the system can achieve high processing accuracy for complex RNNs while optimizing power consumption by adapting to the actual computational requirements of each model
3Adaptability or versatility
If hardware is designed to support multiple RNN variants, then adaptability is improved, but device complexity increases
Solution Approach 1:
The patent implements universal computation blocks that can execute multiple RNN variants through a unified hardware architecture. The computation blocks are designed with generic structures that can be configured to implement different RNN cell types (LSTM, GRU, simple RNN) and variants through parameter settings rather than dedicated hardware for each variant, thereby achieving broad adaptability without proportionally increasing device complexity
Solution Approach 2:
The patent employs dynamic reconfiguration capabilities that allow the same hardware resources to be adaptively allocated to support different RNN variants. The system can dynamically change the operational characteristics of computation blocks to match the requirements of the target RNN variant, enabling a single hardware design to efficiently support multiple variants without the complexity overhead of dedicated hardware for each variant
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Recurrent neural networks are efficiently mapped to hardware computation blocks specifically designed for Legendre Memory Unit (LMU) cells, Projected LSTM cells, and Feed Forward cells. Iterative resource allocation algorithms are used to partition recurrent neural networks and time multiplex them onto a spatial distribution of computation blocks, guided by multivariable optimizations for power, performance, and accuracy. Embodiments of the invention provide systems for low power, high performance deployment of recurrent neural networks for battery sensitive applications such as automatic speech recognition (ASR), keyword spotting (KWS), biomedical signal processing, and other applications that involve processing time-series data.