3D NAND Compute-in-Memory for Neural Network Weight Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks are computationally intensive due to the intensive data transfers involved in reading and transferring weights between memory and processing units, which limits their efficiency.
Innovation Solution
Implement a compute-in-memory approach using non-volatile memory arrays to store weight values in memory cells and apply input values as voltage levels for concurrent matrix multiplication, utilizing a three-dimensional NAND memory architecture with multi-bit values and expanded threshold voltage levels to enhance storage density and computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional memory and processing units are used separately, then data transfer between memory and processing units can be performed, but computational intensity and data transfer overhead increase significantly
Solution Approach 1:
The patent merges memory storage and computation functions into a single integrated structure by implementing neural network weights directly within the three-dimensional NAND memory array. Weight values are encoded as threshold voltage levels in memory cells, allowing multiplication operations to be performed during read operations without separate processing units, thereby eliminating data transfer overhead between memory and processor.
Solution Approach 2:
The patent introduces sense amplifiers as intermediary components that perform multiplication operations during the read process. These sense amplifiers act as mediators between the memory cells storing weights and the output registers, enabling in-memory computation by multiplying input values with stored weight values during the normal read operation without requiring additional data movement.
2Measurement precision
If more weight values are stored in memory to increase neural network complexity, then accuracy improves, but memory capacity requirements increase
Solution Approach 1:
The patent transitions from binary storage to multi-level cell (MLC) storage by utilizing different threshold voltage levels within memory cells to represent multiple weight values simultaneously. This dimensional change from binary (0 or 1) to multi-level (multiple threshold voltage states) allows significantly more weight values to be stored in the same physical memory capacity, enabling more complex neural networks without proportionally increasing memory requirements.
Solution Approach 2:
The patent changes the storage parameter from binary states to continuous threshold voltage levels. By programming memory cells to different threshold voltage levels corresponding to different weight values, the system can store a much larger range of values in the same physical space. This parameter change enables high-precision neural network weights to be stored efficiently in non-volatile memory.
3Quantity of substance
If three-dimensional NAND memory is used instead of traditional memory, then storage density increases, but manufacturing complexity increases
Solution Approach 1:
The patent makes the three-dimensional NAND memory array universally functional by designing it to simultaneously serve as both storage memory and computation unit. The same memory cells that store weight values also perform multiplication operations during read operations. This multi-functionality eliminates the need for separate processing units and reduces overall system complexity despite the increased manufacturing complexity of 3D NAND structures.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach significantly reduces computational intensity by performing in-memory computations efficiently, enhancing accuracy and storage density through multi-level encoding of weights and inputs, thereby optimizing neural network operations.
Implementation Method 1
The memory cells of the sub-array are programmed to different threshold voltage levels corresponding to different weight values of the layer
Implementation Method 2
the memory cells are operated in a linear region and use multi-bit values to expand the number of weight levels
Data Source
AI summary
A non-volatile memory device for performing compute in memory operations for a neural network uses a three dimensional NAND architecture. Multi-bit weight values are stored encoded as sets of threshold voltages for sets of memory cells. A weight value is stored in multiple memory cells on the same word line and connected between a bit line and a source line, each of the memory cells programmed to one of multiple threshold voltages. When multiplying an input value with the weight value, the word line is biased so that, for at least one of the threshold voltages, the memory cell will be in the linear operation region. Input values are encoded as a set of one or more voltage levels applied to a corresponding set of bit lines, each bit line connected memory cells also storing the weight value, connected to the word line, and connected to the source line.