Quantized LSTM Hardware Using NVM Arrays for Low-Bit VMM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions for embedded LSTM neural networks fail to adequately reduce the bit-width of weights and activations, leading to insufficient memory storage size and computation complexity reduction, especially in hardware implementations for mobile devices and self-driving cars.
Innovation Solution
A quantized neural network architecture that converts digital input signals and hidden states into analog signals, using non-volatile memory weight arrays for vector matrix multiplication, and processes these values into new hidden states with reduced bit-width, incorporating digital-to-analog and analog-to-digital converters to maintain performance comparable to 32-bit implementations while providing power-saving advantages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 32-bit precision weight memory arrays are used, then machine learning performance is maintained, but memory size and computation complexity are insufficiently reduced
Solution Approach 1:
The patent changes the precision parameter of weight memory arrays from 32-bit to lower bit-width (e.g., 8-bit, 4-bit, or 1-bit), directly reducing memory size and computation complexity while maintaining acceptable machine learning performance through quantization techniques
Solution Approach 2:
The patent uses lower-precision weight memory arrays that are cheaper and consume less resources, accepting that some precision is sacrificed but overall system performance is maintained through architectural optimizations and quantization-aware training
2Ease of manufacture
If analog VMM is used with limited NVM cell conductance levels, then hardware implementation is achieved, but weight precision is insufficient
Solution Approach 1:
The patent introduces digital-to-analog converters (DACs) and analog-to-digital converters (ADCs) as intermediary components between digital control circuits and analog NVM weight arrays, enabling precise control and readout of analog weights while maintaining hardware efficiency
Solution Approach 2:
The patent segments the computational process into distinct digital and analog domains, with digital circuits handling control and data preparation, analog NVM arrays performing parallel VMM operations, and digital circuits processing results, thereby combining the advantages of both domains
3Measurement precision
If high precision periphery circuits are used, then computation accuracy is maintained, but power consumption and area cost increase
Solution Approach 1:
The patent changes the precision parameter of periphery circuits (DACs, ADCs, activation function units) to match the quantized weight precision, reducing power consumption and area while maintaining overall computation accuracy through coordinated quantization across all components
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The proposed architecture effectively quantizes LSTM neural networks, reducing memory size and computation complexity while maintaining state-of-the-art machine learning performance, enabling offline AI functionality in devices without network connectivity and supporting low-power systems like mobile devices and self-driving cars.
Implementation Method 1
The cross-point structure is inspired by biology where each pre-synaptic neuron corresponds to each row and each post-synaptic neuron corresponds to each column, therefore each cross junction may be one synapse, which is represented by one NVM cell, in the example shown as NVM cell 110, although other variations are possible. When used in the read mode, i.e., the conductance values of the NVM weight cells (Gnm) are stationary, the NVM array can accelerate VMM (Vector-Matrix Multiplications) operations directly in physics using Ohm's law.
Data Source
AI summary
An innovative low-bit-width device may include a first digital-to-analog converter (DAC), a second DAC, a plurality of non-volatile memory (NVM) weight arrays, one or more analog-to-digital converters (ADCs), and a neural circuit. The first DAC is configured to convert a digital input signal into an analog input signal. The second DAC is configured to convert a digital previous hidden state (PHS) signal into an analog PHS signal. NVM weight arrays are configured to compute vector matrix multiplication (VMM) arrays based on the analog input signal and the analog PHS signal. The NVM weight arrays are coupled to the first DAC and the second DAC. The one or more ADCs are coupled to the plurality of NVM weight arrays and are configured to convert the VMM arrays into digital VMM values. The neural circuit is configured to process the digital VMM values into a new hidden state.