Charge-Summing Memory Cell Strings for In-Memory Matrix-Vector Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-memory compute devices, such as those based on NAND flash memory, lack compactness and efficiency for performing matrix-vector multiplications, leading to energy inefficiencies and reliability issues during neural network computations.
Innovation Solution
A dense in-memory compute device with serially connected memory cells over a semiconductor channel structure, utilizing programmable threshold transistors for weight storage and a readout circuit to buffer and convert charge packets into output voltages, allowing for energy-efficient matrix-vector multiplications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If NAND flash memory arrays are used for in-memory compute, then device density is improved, but the device lacks compactness and energy efficiency for matrix-vector multiplications
Solution Approach 1:
The patent merges the storage function (NAND flash memory cells storing weights) with the computation function (matrix-vector multiplication) into a single integrated device. The memory cell strings serve both as weight storage elements and as the computational substrate, eliminating the need for separate storage and compute units. This merging enables in-situ computation where multiply-and-accumulate operations are performed directly within the memory array, dramatically improving energy efficiency by avoiding data movement between storage and compute units.
Solution Approach 2:
The memory cell strings are designed to perform multiple functions: storing weight values, performing analog multiply operations with input signals, accumulating partial products, and generating output signals. The same physical structure (memory cell string with transistors and charge storage) is utilized for both data retention and computational processing, making the device universally capable of both storage and compute operations.
2Measurement precision
If output currents are sensed in existing in-memory compute devices, then computation results are obtained, but reliability issues occur during neural network computations
Solution Approach 1:
The patent substitutes the conventional current sensing mechanism with a voltage-based readout mechanism. Instead of measuring output currents that flow through the memory cell strings (which causes read disturb and reliability issues), the invention uses charge-to-voltage conversion at sense nodes. The accumulated charge from the multiply-and-accumulate operations is converted to voltage signals that can be read out without drawing significant current through the memory cells, thereby eliminating the reliability problems associated with current sensing.
3Productivity
If frequent data accesses to off-chip memory are performed, then neural network computations are executed, but energy bottlenecks occur
Solution Approach 1:
The patent performs preliminary action by pre-storing the weight matrix values within the memory array before computation is needed. The weight values are programmed into the memory cells in advance, so that during inference operations, only the input vectors need to be supplied. The computation is then performed in-situ using these pre-loaded weights, eliminating the need for frequent back-and-forth data transfers between off-chip memory and compute units during the actual multiplication operations.
Solution Approach 2:
The memory array serves itself by performing the multiplication operations internally without requiring external compute units. The memory cells themselves conduct the computational work by utilizing their natural electrical characteristics (threshold voltages, charge storage) to perform analog multiply and accumulate operations. This self-service capability eliminates the need for separate processing units and the associated data movement, dramatically reducing energy consumption.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The device achieves near-maximum density and energy efficiency for matrix-vector multiplications, supports various neural network architectures, and reduces reliability issues by summing charge packets without sensing output currents, enhancing prediction performance.
Implementation Method 1
Each memory cell comprises a programmable threshold transistor adapted for permanently storing a threshold voltage representing a respective weight of the set of weight inputs
Implementation Method 2
the readout circuit is configured to buffer transferred charge packets from the string of memory cells as a charge sum signal present on the sense node
Implementation Method 3
applying a pass mode signal to control gate terminals of the respective memory cells of the string to cause the memory cells of the string to be switched on
Implementation Method 4
enabling the charge transfer switch to connect the sense node to the string of memory cells while applying the stop signals
Data Source
Figure 1~2
Figure 3~5
Figure 6~8
AI summary
An in-memory compute device for multiply-and-accumulate operations and methods of operating the same. The device comprises a string of series-connected memory cells (11a-11e) formed over a semiconductor channel structure (12). Each memory cell comprises a programmable threshold transistor to store a weight input. A readout circuit (15) includes a sense node (14) for buffering charge packets transferred from the string of memory cells. Control circuitry is configured for applying pass mode signals, data input signals and stop signals to the control gates of the respective memory cells. Pass mode signals cause memory cells of the string to be switched on, thereby inducing charge packets in the channel structure underneath each memory cell. Data input signals representing binary zeroes and stop signals cause memory cells of the string to be switched off. Data input signals and stop signals are applied sequentially according to each memory cell's position along the string.