Charge-Summing Memory Cell Strings for Energy-Efficient MACs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional Von Neumann computer architectures face energy inefficiencies due to frequent data accesses to off-chip memory for deep neural network computations, and existing in-memory compute devices lack compactness and reliability for matrix-vector multiplications.
Innovation Solution
A dense in-memory compute device with serially connected memory cells over a semiconductor channel structure, using programmable threshold transistors for weight storage and a readout circuit to buffer and convert charge packets into output voltages, allowing for energy-efficient matrix-vector multiplications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If traditional Von Neumann architecture is used for deep neural network computations, then computation can be performed with floating-point precision, but energy efficiency deteriorates due to frequent data accesses to off-chip memory
Solution Approach 1:
The patent merges memory storage and computation functions into a single integrated structure. Memory cells store weight values while simultaneously participating in analog multiply-and-accumulate operations, eliminating the need for separate memory access operations and reducing energy consumption from data transfers between memory and processing units
Solution Approach 2:
The patent introduces analog voltage signals as intermediaries to represent data values during computation. Instead of digital readout and processing, analog voltages directly encode weight and input values, enabling computation to occur in the analog domain within the memory array itself, thereby avoiding energy-intensive digital-to-analog and analog-to-digital conversions
2Use of energy by moving object
If existing in-memory compute devices are used for matrix-vector multiplications, then energy efficiency is improved, but device compactness and reliability deteriorate
Solution Approach 1:
The patent segments the computation process into distinct phases: weight storage in memory cells, analog voltage application for inputs, charge transfer for multiplication, and digital readout for accumulation. This segmentation allows each phase to be optimized independently, improving overall reliability while maintaining energy efficiency
Solution Approach 2:
The patent replaces traditional digital computing mechanics with analog charge-based computation. Instead of using digital logic gates and sequential processing, the system uses analog voltage signals and charge transfer mechanisms to perform multiply-and-accumulate operations, improving both energy efficiency and reliability by reducing the number of discrete operation steps
3Area of stationary object
If dense NAND flash memory arrays are used for in-memory computing, then compactness is improved, but the ability to perform analog multiply-and-accumulate operations deteriorates
Solution Approach 1:
The patent makes dense NAND flash memory cells universal by enabling them to perform multiple functions: storing weight values digitally, participating in analog voltage division, transferring charges proportionally to weight values, and contributing to accumulated sums. This multi-functionality allows the same compact memory structure to serve both storage and computation purposes without sacrificing either capability
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution achieves highly energy-efficient matrix-vector multiplications with improved prediction performance and reduced reliability issues, enabling on-chip storage of weight matrices for various neural network architectures without external memory access.
Implementation Method 1
Each memory cell comprises a programmable threshold transistor adapted for permanently storing a threshold voltage representing a respective weight of the set of weight inputs
Implementation Method 2
The readout circuit is configured to buffer transferred charge packets from the string of memory cells as a charge sum signal present on the sense node and convert the charge sum signal into an output voltage
Data Source
AI summary
Example embodiments relate to matrix-vector multiplications based on charge-summing memory cell strings. An example in-memory compute device for performing analog multiply-and-accumulate operations on a set of data inputs and a set of weight inputs includes a string of serially connected memory cells formed over a semiconductor channel structure, a source junction controllably connectible to one end of the string of memory cells via a string select switch, a readout circuit including a sense node controllably connectible to one end of the string of memory cells via a charge transfer switch, and control circuitry. The control circuitry is configured to apply pass mode signals, data input signals, and stop signals sequentially according to each memory cell's position along a string. The control circuitry is also configured to enable the string select switch and the charge transfer switch.