NVM Memory Banks for Efficient Neural Turing Machines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing recurrent neural networks (RNNs), particularly Neural Turing Machines (NTMs), face inefficiencies in processing due to reliance on volatile random access memory (RAM) which limits computational efficiency and increases power consumption.
Innovation Solution
Implementing NTM architectures using non-volatile memory (NVM) arrays with on-chip computational components to store and process NTM matrices, optimizing matrix size based on storage access granularity and employing threshold-based read/write operations to reduce unnecessary memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If volatile random access memory (RAM) is used to store NTM matrices, then fast access speed is achieved, but power consumption increases and computational efficiency is limited
Solution Approach 1:
The patent replaces volatile RAM with non-volatile memory (NVM) technology to store NTM matrices. This substitution maintains fast access speeds while significantly reducing power consumption because NVM does not require continuous power to maintain data state, eliminating the need for refresh operations that consume power in volatile memory systems.
Solution Approach 2:
The patent changes the fundamental parameter of memory volatility from volatile to non-volatile. By using NVM with appropriate endurance characteristics, the system achieves both fast access speeds comparable to RAM and ultra-low power consumption, as the memory retains data without power and only consumes energy during write operations.
2Loss of information
If full NTM matrix is read from NVM array, then complete computational data is obtained, but memory access time increases due to sequential reading requirement
Solution Approach 1:
The patent extracts and processes only the relevant portions of the NTM matrix that exceed a threshold value. Instead of reading the entire matrix from NVM, the system identifies and processes only the significant weight values, thereby reducing memory access time while maintaining computational accuracy by excluding negligible values.
Solution Approach 2:
The patent applies partial action by reading only the necessary subset of matrix data rather than the complete matrix. By using threshold-based filtering, the system performs partial reads of the NVM array, accessing only those memory locations containing weight values above the threshold, thus reducing access time while preserving computational integrity.
3Loss of energy
If threshold-based filtering is applied to reduce memory access, then power consumption decreases, but computational precision may be affected
Solution Approach 1:
The patent optimizes the threshold parameter to balance power consumption and computational precision. By carefully selecting the threshold value, the system filters out only those weight values that are truly negligible, ensuring that the remaining values contribute meaningfully to the computation. This parameter optimization maintains precision while achieving significant power savings.
Solution Approach 2:
The patent replaces traditional dense matrix multiplication with a threshold-based sparse computation approach. This substitution allows the system to skip multiplication operations for sub-threshold values, reducing both power consumption and computational load while maintaining precision for the significant values that are processed.
4Use of energy by moving object
If NVM array is used to store RNN memory bank, then power consumption is reduced, but storage access granularity constraints apply
Solution Approach 1:
The patent segments the NVM array into distinct regions: a dedicated RNN memory bank for storing recurrent neural network data and a separate NTM matrix storage area. This segmentation allows independent optimization of access patterns for each region, managing the granularity constraints by organizing data according to its access requirements while maintaining the power benefits of NVM.
Solution Approach 2:
The patent implements dynamic access strategies that adapt to the specific requirements of RNN operations. By dynamically adjusting read/write patterns and utilizing the full-word access capability of NVM, the system optimizes performance for recurrent neural network workloads while managing the inherent granularity constraints of non-volatile memory technology.
Data Source
AI summary
Recurrent Neural Networks (RNNs) wherein a non-volatile memory (NVM) array provides a memory bank for the RNN. The RNN may be a Neural Turing Machine (NTM) and the memory bank may be an NTM matrix stored in the NVM array. In some examples, an NTM controller sets the size of the NTM matrix based on a storage access granularity of the NVM array. For instance, if the NVM reads and writes data in flash memory unit (FMUs), the NTM controller sets the size of the NTM matrix to correspond to the size of an integer number of FMUs. In some examples, the NVM array includes on-chip NTM circuitry configured to perform at least some NTM read head and write head operations. Threshold-based processing is described that can reduce an amount of NTM data read from the NVM array. In other examples, volatile memory is employed rather than an NVM array.