Bonded Memory Device for In-Memory Matrix Vector Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited memory bandwidth and power consumption issues in deep learning applications, particularly in energy-constrained systems, due to the bottleneck of inter-chip data movement and the inefficiency of conventional Von-Neumann computer architecture, hinder the processing of large DNN models with high accuracy and speed.
Innovation Solution
An integrated memory device that combines memory and processing, using a 3D memory array with analog capabilities for matrix vector multiplication and accumulation operations, where memory cells store weights and inputs, and voltage drivers apply read voltages to perform multiplications and summations efficiently, reducing the need for digital logic and minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted from sensors to general-purpose microprocessors for processing, then computation can be performed, but transmission time and power consumption increase
Solution Approach 1:
The patent merges memory and processing units into a single integrated device, allowing computation to be performed directly on data stored in memory without external data transmission. The memory cell array stores input data and weight values, while voltage drivers and sense amplifiers perform multiplication and accumulation operations in-place, eliminating the need to transfer data between separate memory and processor components.
Solution Approach 2:
The patent introduces sense amplifiers as intermediary components that directly read from memory cells and perform computation operations. These sense amplifiers act as mediators between the stored data and the computation logic, enabling in-memory processing by directly utilizing the electrical states of memory cells for multiplication and accumulation without requiring data to be moved to external processors.
2Productivity
If data is transmitted from sensors to microprocessors for processing, then computation can be performed, but power consumption increases
Solution Approach 1:
The patent combines memory storage and processing functions into a single integrated device, eliminating the energy-consuming data transmission process between separate memory and processor components. The memory cell array stores both input data and weight values, and voltage drivers apply read voltages to perform multiplication operations directly within the memory structure, significantly reducing overall power consumption compared to conventional architectures.
Solution Approach 2:
The patent enables the memory system to perform computation operations on its own stored data without requiring external processors. The sense amplifiers and voltage drivers within the memory device itself carry out multiplication and accumulation operations using the electrical states of memory cells, allowing the memory system to serve its own processing needs and eliminating the energy overhead of external data movement and processing.
3Device complexity
If conventional Von-Neumann architecture is used, then system simplicity is maintained, but memory bandwidth is limited
Solution Approach 1:
The patent merges memory and processing functions into a single integrated device, fundamentally changing the data flow architecture. Instead of the conventional Von-Neumann bottleneck where data must be repeatedly moved between memory and processor, the integrated device allows computation to be performed directly on data stored in the memory cell array, with voltage drivers applying read voltages and sense amplifiers reading and accumulating results in-place, thereby dramatically increasing effective memory bandwidth for computation workloads.
4Productivity
If digital logic circuits are used for multiplication, then computation can be performed, but device complexity and power consumption increase
Solution Approach 1:
The patent replaces complex digital logic circuits with a simplified electrical measurement and accumulation system. Instead of using digital multipliers and adders that require numerous logic gates and transistors, the invention uses voltage drivers to apply read voltages to memory cells storing weight values, sense amplifiers to read the resulting currents that represent multiplication results, and accumulators to sum these results. This substitution of electrical measurement for digital logic computation significantly reduces device complexity while maintaining multiplication capability.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables efficient matrix vector multiplication and accumulation operations, reducing power consumption and latency, and enhancing the performance of deep learning applications by integrating memory and processing within the same integrated circuit device.
Implementation Method 1
a memory cell array having a plurality of memory cells, each of the memory cells having a charge storage node
Implementation Method 2
a passivation layer surrounding the charge storage node, the passivation layer preventing the charge storage node from trapping charge
Data Source
AI summary
Systems, methods, and apparatus related to memory devices that perform multiplication using memory cells. In one approach, a first integrated circuit die has a memory cell array. The memory cell array includes memory cells programmable to store weights (e.g., representing synapses of a neural network). A second integrated circuit die has logic circuitry that performs multiplication of the stored weights by an input pattern. The second die is connected to the first die by hybrid bonding. Multiplication results are determined by the logic circuitry based on accumulation of output currents from at least a portion of the memory cells.