3D NAND Flash Multi-Layer VMM for DNN In-Memory Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face challenges in power efficiency and energy consumption due to high demands on memory storage and computational resources, particularly in vector-matrix multiplication (VMM) operations, which are computation-intensive and often bottlenecked by frequent data access from off-chip memory, limiting the implementation of large-scale DNNs on conventional hardware.
Innovation Solution
A multi-layer vector-matrix multiplication apparatus utilizing a three-dimensional (3D) NAND flash structure with multiple transistor array layers, where each layer stores weight matrices and receives selection voltages to perform VMM operations, reducing the footprint and enabling in-memory computing for DNNs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If eNVM-based crossbar array architecture is used to implement VMM, then data transfer is reduced, but computation accuracy is compromised due to I-V nonlinearity
Solution Approach 1:
The patent changes the electrical parameters by using selector devices with specific threshold voltages to control current flow paths. The selector devices are configured with threshold voltages that enable linear current-voltage characteristics during VMM operations, thereby maintaining computation accuracy while still benefiting from in-memory computing architecture.
Solution Approach 2:
The patent introduces selector devices as intermediary components between the eNVM crossbar array and the external interface. These selector devices act as mediators that filter out non-linear effects and provide linear current responses, thus preserving computation accuracy while enabling energy-efficient in-memory computing.
2Productivity
If eNVM-based crossbar array is used, then in-memory computing is enabled, but density is limited due to sneak path effect
Solution Approach 1:
The patent extracts and eliminates the harmful sneak path currents by introducing selector devices that block these unwanted current paths. By removing the sneak path effect, the crossbar array can be configured with higher density without suffering from computational errors, thus enabling both in-memory computing and high density.
Solution Approach 2:
The patent segments the crossbar array into smaller functional units with selector devices at strategic locations. This segmentation allows independent control of different regions, enabling high-density configuration while maintaining computational accuracy by isolating sneak path effects to localized areas that can be managed individually.
3Area of stationary object
If conventional 3D NAND flash is reconfigured for VMM, then footprint is reduced, but computation accuracy may be affected by I-V nonlinearity
Solution Approach 1:
The patent modifies the operational parameters of the 3D NAND flash by applying specific voltage sequences and utilizing threshold voltage characteristics of the transistors. By carefully controlling the voltage parameters and timing, the system achieves linear current responses from the NAND flash cells, maintaining computation accuracy while benefiting from the small footprint of 3D NAND technology.
Solution Approach 2:
The patent makes the 3D NAND flash structure universal by enabling it to perform both data storage and VMM operations. Through careful configuration and control, the same physical structure serves multiple functions, achieving compact footprint while maintaining computational accuracy through programmable voltage control and multi-functional transistor operation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The 3D NAND flash structure supports multi-layer VMM operations with a smaller footprint, enhancing the power efficiency and computational capabilities for deep neural networks by performing arithmetic computations closer to data storage, thus reducing data transfer and improving computation accuracy.
Implementation Method 1
each of the transistor array layers can perform a respective VMM operation by multiplying the respective selection voltages with the respective weight matrix
Data Source
AI summary
A multi-layer vector-matrix multiplication (VMM) apparatus is provided. The multi-layer VMM apparatus includes a three-dimensional (3D) NAND flash structure having multiple transistor array layers each includes a number of transistors configured to store a respective weight matrix and a number of word lines configured to receive respective selection voltages corresponding to a respective input vector. Accordingly, each of the transistor array layers can perform a respective VMM operation by multiplying the respective selection voltages with the respective weight matrix. Thus, by providing the respective selection voltages to each of the multiple transistor array layers in a sequential order, it may be possible to carry out a multi-layer VMM operation in the 3D NAND flash structure with reduced footprint, thus making it possible to support a deep neural network (DNN) via such advanced techniques as in-memory computing.