Memory Device Signed Multiplication 3D Array
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited memory bandwidth and power consumption issues in deep learning applications, particularly in edge AI systems, due to the bottleneck of inter-chip data movement and the inefficiency of conventional Von-Neumann computer architecture, where frequent data movement between compute units and memory leads to latency and bandwidth limitations.
Innovation Solution
An integrated memory device that combines memory and processing, using a 3D memory array with analog capability to perform matrix vector multiplication and accumulation operations efficiently by programming memory cells to represent weights and applying voltages to achieve parallelized operations, reducing the need for digital logic gates and minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted from sensors to general-purpose microprocessors for processing, then computation can be performed, but transmission bandwidth is limited and power consumption increases
Solution Approach 1:
The patent combines memory and processing units into a single integrated device, allowing computation to be performed directly where data is stored. This eliminates the need for data transmission between separate memory and processor components, thereby reducing power consumption while maintaining computation performance.
Solution Approach 2:
The integrated memory device performs multiple functions including data storage, matrix-vector multiplication, and accumulation operations within the same device. This multi-functionality eliminates the need for separate processing units, reducing overall system power consumption while maintaining high computation performance for AI workloads.
2Productivity
If frequent data movement occurs between compute units and memory, then computation can be performed, but latency increases and bandwidth is limited
Solution Approach 1:
By integrating the processing unit directly within the memory device, the patent eliminates data movement between separate memory and processor components. This reduces both latency and bandwidth limitations, enabling high-throughput computation for AI workloads without the time penalties associated with frequent data transfer.
Solution Approach 2:
The processing unit is divided into multiple parallel computation units that can simultaneously perform matrix-vector multiplication and accumulation operations on different data segments. This segmentation enables parallel processing, increasing computation throughput while minimizing latency through concurrent operations.
3Productivity
If digital logic gates are used for multiplication operations, then computation can be performed, but power consumption increases
Solution Approach 1:
The patent replaces traditional digital logic gate-based multiplication with analog computation using resistive memory cells. The multiplication operation is performed through physical resistor network calculations, which consume significantly less power than digital logic gates while maintaining high computation speed for matrix-vector operations.
Solution Approach 2:
The patent changes the operational parameters from digital voltage levels to analog resistance values. By programming memory cells to represent weights as resistance values and using voltage division in resistor networks, the system achieves high-speed multiplication with reduced power consumption compared to digital logic implementations.
4Device complexity
If conventional Von-Neumann architecture is used, then system design is simplified, but inter-chip data movement creates bandwidth bottlenecks
Solution Approach 1:
The patent merges memory and processing functions into a single integrated device, eliminating the separation inherent in conventional Von-Neumann architecture. This reduces inter-chip data movement and bandwidth consumption while the modular design maintains relative simplicity in system integration.
Solution Approach 2:
The patent transitions from a two-dimensional Von-Neumann architecture with separate memory and processor to a three-dimensional integrated structure where processing units are embedded within the memory device. This dimensional change enables closer integration, reducing data movement distances and bandwidth requirements while maintaining manageable system complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables more efficient matrix vector multiplication and accumulation operations, reducing power consumption and latency, and improving the performance of AI applications by integrating memory and processing in the same integrated circuit device.
Implementation Method 1
When a voltage representative of a second input bit is applied on the memory cell, an amount of current output by the memory cell is representative of a result of the multiplication
Data Source
AI summary
Systems, methods, and apparatus related to memory devices that perform multiplication using sets of four memory cells. In one approach, memory cells in a memory cell array are programmed so that each set stores a signed weight. Voltages are applied to the sets of memory cells. The voltages represent signed inputs to be multiplied by the signed weights. Output currents from the memory cells in each set are summed in first and second lines. A sum of the output currents in each line is digitized to provide first and second results. The first and second results are combined to provide a signed result for each set.