3D Memory Array for Matrix Vector Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited memory bandwidth and power consumption issues in deep learning applications, particularly in energy-constrained systems, due to the bottleneck of inter-chip data movement in conventional Von-Neumann computer architecture, hinder the efficiency of AI tasks like image processing and neural network computations.
Innovation Solution
An integrated memory device that combines memory and processing in a single integrated circuit, using a 3D memory array with analog capabilities for matrix vector multiplication and accumulation operations, reducing the need for digital logic and minimizing power consumption by operating memory cells in a sub-threshold mode.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted from sensors to general-purpose microprocessors for processing, then processing capability is provided, but transmission efficiency deteriorates and power consumption increases
Solution Approach 1:
The patent merges memory and processing functions into a single integrated circuit device. The memory cell array performs multiplication and accumulation operations directly during data reading, eliminating the need to transmit data to a separate microprocessor. This integration resolves the contradiction by providing processing capability at the memory level, thereby improving transmission efficiency and reducing power consumption.
Solution Approach 2:
The patent replaces the conventional mechanical data transmission and separate processing approach with an analog computation system. Memory cells operate in sub-threshold mode to perform multiplication and accumulation operations directly on read data, substituting the need for digital signal transmission and separate processing units. This substitution improves efficiency while reducing energy consumption.
2Productivity
If conventional Von-Neumann architecture is used, then processing can be performed by microprocessor, but memory bandwidth is limited due to inter-chip data movement bottleneck
Solution Approach 1:
The patent combines memory storage and computation processing within a single integrated circuit. The memory cell array is configured to perform multiplication and accumulation operations during the data reading process itself, eliminating the separate data movement phase. This merging of functions directly increases memory bandwidth and reduces data movement latency.
Solution Approach 2:
The patent performs computation operations preliminarily during the data reading phase. By configuring memory cells to execute multiplication and accumulation operations as data is read from storage, the system completes computations before data needs to be transmitted to external processing units. This preliminary action eliminates the bottleneck of inter-chip data movement and improves overall memory bandwidth.
3Productivity
If specialized multiplier-accumulator circuits are used, then computation performance is improved, but device complexity increases
Solution Approach 1:
The patent makes the memory cell array universal by enabling it to perform multiple functions: data storage, multiplication, and accumulation operations. Each memory cell operates in sub-threshold mode to execute computation functions, eliminating the need for separate specialized multiplier-accumulator circuits. This multi-functionality approach improves computation performance while reducing overall device complexity.
Solution Approach 2:
The memory cells perform computation operations autonomously during data reading without requiring external specialized processing circuits. The memory cell array serves itself by executing multiplication and accumulation functions as part of its normal operation, thereby improving computation performance while avoiding the complexity of separate dedicated hardware units.
4Loss of energy
If memory cells operate in sub-threshold mode, then power consumption is reduced, but manufacturing precision requirements increase
Solution Approach 1:
The patent changes the operating parameters of memory cells by operating them in sub-threshold mode rather than conventional threshold mode. This parameter change reduces power consumption while maintaining computational functionality. The system manages the increased manufacturing precision requirements through careful circuit design and operational protocols that compensate for threshold voltage variations.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables efficient performance of matrix vector multiplication and accumulation operations, reducing power consumption and latency, and enhancing the processing efficiency for AI applications by integrating memory and processing, thus addressing the memory bandwidth and power consumption challenges.
Implementation Method 1
applying voltages to the memory cells for performing the multiplication, the voltages being applied so that operation of the memory cells remains in a sub-threshold mode during the multiplication
Implementation Method 2
at least one line coupled to the memory cells, the line being configured to sum output currents from each of the memory cells
Data Source
AI summary
Systems, methods, and apparatus related to memory devices that perform multiplication using sets of memory cells. In one approach, memory cells in the sets are programmed so that each set stores a signed weight. Voltage drivers apply voltages to the memory cells in each set. The voltages correspond to signed inputs to multiply by the signed weights in the sets. One or more common lines (e.g., bitlines) are coupled to each set for summing output currents from the sets. A digitizer provides a signed result based on summing the output currents from the sets.