3D NOR Memory Multiply-Accumulate Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer architectures face limitations in memory bandwidth and power consumption due to the bottleneck at the interface between processor chips and DRAMs, particularly in deep learning systems that require large amounts of data fetching and storing.
Innovation Solution
A three-dimensional memory cell array in a NOR configuration is used within memory devices to perform multiplication and other operations, allowing weights to be stored and multiplied within the memory device itself, reducing the need for data movement to processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If DRAM devices are used to store large amounts of weights and activations, then storage density is improved, but memory bandwidth and power consumption deteriorate due to constant data fetching between processor and memory
Solution Approach 1:
The patent combines memory storage and multiplication operations into a single integrated device. The three-dimensional NOR memory device stores weights in memory cells and performs multiplication operations within the same device, eliminating the need for separate processor-memory interfaces and reducing data movement between components.
Solution Approach 2:
The patent employs three-dimensional vertical stacking of memory cells to increase storage density within the same footprint. By stacking memory cells vertically in multiple tiers, the device achieves higher storage capacity while maintaining compact form factor and reducing the physical distance for data access.
2Quantity of substance
If DRAM devices are used for high density storage, then storage capacity is improved, but memory bandwidth deteriorates due to interface bottleneck between processor and memory
Solution Approach 1:
The patent merges storage and computation functions into a single integrated device. By performing multiplication operations directly within the memory device using stored weights and input signals, the system eliminates the bandwidth bottleneck caused by constant data transfer between separate processor and memory components.
Solution Approach 2:
The memory device performs multiplication operations autonomously using its own stored weights and received input signals, without requiring external processor intervention for each operation. This self-service capability reduces the burden on the processor-memory interface and improves effective bandwidth.
3Adaptability or versatility
If data is constantly fetched from and stored in external DRAM, then neural network processing is enabled, but training time deteriorates due to repeated loading and saving operations
Solution Approach 1:
The patent integrates storage and multiplication operations to enable neural network processing within a single device. By maintaining weights locally in the three-dimensional NOR memory and performing computations in-place, the system eliminates repeated data loading and saving operations that slow down training.
Solution Approach 2:
The weights are pre-loaded into the three-dimensional NOR memory device once, and then the device autonomously performs multiple multiplication operations using these stored weights. This preliminary action of loading weights once eliminates the need for repeated loading during training iterations.
4Adaptability or versatility
If processor chips and DRAMs are used in current architecture, then computational capability and storage capacity are achieved, but power consumption increases due to overhead at the interface between devices
Solution Approach 1:
The patent combines memory storage and multiplication operations into a single integrated device, eliminating the processor-memory interface that causes power consumption overhead. The three-dimensional NOR memory device performs both storage and computation functions, removing the need for continuous data transfer between separate components.
Solution Approach 2:
The memory device performs multiplication operations autonomously using its own internal resources (stored weights and input signals), without requiring external processor intervention. This self-service approach eliminates the power-consuming interface operations between separate processor and memory devices.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances power efficiency and reduces memory bandwidth issues, enabling faster training times and lower power consumption in neural networks, especially in mobile and automotive applications.
Implementation Method 1
A three-dimensional memory cell array in a NOR configuration is used within memory devices to perform multiplication and other operations
Data Source
AI summary
Systems, methods, and apparatus related to memory devices that perform multiplication using memory cells. In one approach, a memory cell array has memory cells stacked vertically above a semiconductor substrate. Each memory cell stores a weight. Local digit lines connect to terminals of the memory cells. The local digit lines extend vertically above the substrate. Select transistors connect to the local digit lines. Select lines control the select transistors, and are used to encode an input pattern to multiply by the stored weights. Accumulation circuitry sums output currents from the memory cells. In one example, each memory cell is formed using a transistor that includes a semiconductor layer to provide a horizontal channel. A gate layer (e.g., a gate stack layer) wraps around a circumference of the semiconductor layer. Wordlines apply gate voltages to the transistors. Each wordline has a respective portion that wraps around a circumference of the gate layer of each transistor.