Context-Based 3D Memory Array Programming for In-Memory MVM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Limited memory bandwidth and high power consumption in machine learning systems, particularly in neural networks, due to the bottleneck at the interface between processor chips and DRAMs, leading to latency and inefficient data movement.
Innovation Solution
Integration of memory and processing in an integrated circuit device using multi-pillar memory cells to perform matrix vector multiplication directly in the memory array, reducing IR drops and improving computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If memory and processing are separated into different chips (processor chip and DRAM), then specialized processing can be achieved, but bandwidth limitations and power consumption increase due to the interface bottleneck
Solution Approach 1:
The patent merges memory and processing functions into a single integrated circuit device. The memory array directly performs matrix-vector multiplication operations without requiring data transfer to a separate processor chip. This integration eliminates the interface bottleneck between processor and memory, reducing both power consumption and bandwidth limitations while maintaining specialized processing capability for neural network computations.
2Adaptability or versatility
If memory and processing are separated into different chips, then processing specialization is improved, but computational efficiency decreases due to latency and bandwidth limitations
Solution Approach 1:
The patent combines memory storage and processing operations within the same integrated circuit device. The memory array is configured to perform matrix-vector multiplication directly on stored data, eliminating the need to transfer data between separate memory and processor chips. This integration removes latency and bandwidth constraints, significantly improving computational efficiency for neural network operations.
3Measurement precision
If memory cells are programmed to precise target currents, then computational accuracy is improved, but programming time increases due to iterative adjustments
Solution Approach 1:
The patent applies context-based pre-adjustment to programming parameters before the actual programming process. By determining contextual information about the memory array (such as temperature, process variations, or array-specific characteristics) and using this context to pre-calculate optimal programming parameters, the system achieves precise target currents faster. This preliminary action reduces the number of iterative adjustments needed, thereby reducing programming time while maintaining computational accuracy.
4Measurement precision
If iterative adjustments are made to reach target currents, then programming precision is improved, but power consumption increases
Solution Approach 1:
The patent uses context information to pre-determine optimal programming parameters before initiating the programming process. By calculating appropriate programming voltages and currents based on contextual factors (such as array characteristics or environmental conditions), the system achieves the desired programming precision in fewer iterative steps. This reduction in iteration count directly lowers the power consumption associated with repeated programming attempts while maintaining high programming precision.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances computational efficiency and reduces power consumption by performing matrix vector multiplication directly in the memory array, thereby addressing bandwidth limitations and power inefficiencies.
Implementation Method 1
A bitline is electrically connected to the first and second pillars, and the bitline is configured to accumulate output currents from the first and second pillars when performing multiplication
Data Source
AI summary
Systems, methods, and apparatus for memory devices. In one approach, a memory device has memory cells arranged in a three-dimensional vertical memory array. The memory cells are accessed using bitlines that are formed overlying the array. A controller determines a context of the memory cells. Based on the context, each memory cell is programmed to have an output current that corresponds to a stored weight. Output currents from the memory cells are accumulated using the bitlines for performing matrix vector multiplication.