3D NOR Compute-in-Memory for Low-Power Neural Network Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks are computationally intensive and require significant data transfers to read and transfer weight values between memory and processing units, which can be power and time intensive even with compute-in-memory approaches.
Innovation Solution
The use of three-dimensional NOR memory structures operating in the subthreshold region, where weight values are encoded as threshold voltages and activations are encoded as word line voltages, allowing for efficient compute-in-memory operations with low power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weights are stored in non-volatile memory and read into processing units, then neural network computations can be performed, but data transfer between memory and processing units becomes power and time intensive
Solution Approach 1:
The patent combines memory and processing functions into a single integrated structure where NOR memory cells perform both data storage and neural network computation. The memory array itself executes matrix multiplication operations by applying input voltages to word lines and sensing output currents from bit lines, eliminating the need for separate data transfer between memory and processing units.
Solution Approach 2:
The patent introduces control circuitry as an intermediary that manages the dual function of the NOR memory array. This control circuitry coordinates the application of input voltages, monitors output currents, and handles data read/write operations, enabling the memory structure to perform compute-in-memory operations without requiring traditional processing units.
2Use of energy by moving object
If three-dimensional NOR memory structures are used for compute-in-memory operations, then power consumption is reduced, but device complexity increases
Solution Approach 1:
The NOR memory array is designed to serve multiple functions: it stores weight values in its memory cells, performs matrix multiplication computations by applying input voltages to word lines, and outputs computation results through bit line currents. This multi-functionality reduces the need for separate dedicated processing circuits, thereby managing complexity while achieving low-power operation.
Solution Approach 2:
The patent transitions from two-dimensional NOR memory to three-dimensional NOR memory structures. This vertical stacking increases storage density and computational capacity within a smaller footprint, allowing more neural network layers and parameters to be implemented without proportionally increasing power consumption or overall device complexity.
3Quantity of substance
If large neural networks are implemented with compute-in-memory operations, then bandwidth requirements are reduced, but manufacturing precision requirements increase
Solution Approach 1:
The patent utilizes the threshold voltage parameter of NOR memory cells to encode weight values for neural networks. By programming different threshold voltages into the memory cells, the system can represent a range of weight values without requiring additional circuitry. This parameter-based encoding allows large neural networks to be implemented while maintaining manufacturing feasibility, as threshold voltage programming is a standard memory operation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables high-density, low-power neural network computations, efficiently implementing large networks through compute-in-memory operations while reducing bandwidth and power requirements.
Implementation Method 1
weight values are encoded as threshold voltages
Implementation Method 2
operating in the subthreshold region
Implementation Method 3
three-dimensional NOR memory structures
Data Source
AI summary
A non-volatile memory device for performing compute in memory operations for a neural network uses a three dimensional NOR architecture in which vertical NOR strings are formed of multiple memory cells connected in parallel between a source line and a bit line. Weights of the neural network are encoded as threshold voltages of the memory cells and activations are encoded as word line voltages applied to the memory cells of the NOR strings. The memory cells are operated in the subthreshold region, where the word line voltages are below the threshold voltages. The NOR structure naturally sums the resultant subthreshold currents of the individual memory cells to generate the product of the activations and the weights of the neural network by concurrently applying input voltages to multiple memory cells of a NOR string.