NAND Memory Array Biasing for In-Memory Matrix-Vector Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Limited memory bandwidth and high power consumption are significant challenges in machine learning systems, particularly in deep neural networks, due to the bottleneck at the interface between processor chips and DRAMs, leading to latency and inefficient data movement.
Innovation Solution
Integrate memory and processing in a single integrated circuit device using multi-pillar memory cells to perform matrix vector multiplication directly in the memory array, reducing IR drops and improving computational efficiency by parallelizing operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in external DRAM and processed by GPU/CPU, then storage capacity is sufficient, but memory bandwidth is limited and power consumption increases
Solution Approach 1:
The patent merges memory storage and processing functions into a single integrated circuit device. The memory cell array simultaneously stores weight data and performs multiplication operations, eliminating the need for separate external DRAM and GPU/CPU components. This consolidation reduces power consumption by eliminating data movement between separate components while maintaining sufficient storage capacity for neural network operations.
2Quantity of substance
If data is stored in external DRAM, then storage density is high, but memory bandwidth is limited causing latency
Solution Approach 1:
The patent combines memory and processing in a single integrated device, allowing weight data to be stored in the memory cell array and immediately processed through multiplication operations within the same device. This eliminates the memory bandwidth bottleneck by removing the need for data to be transferred between external DRAM and processing units, thereby increasing operational speed while maintaining storage density.
3Productivity
If memory and processing are integrated in a single device, then computational efficiency improves, but device complexity increases
Solution Approach 1:
The patent implements a universal memory cell array that performs both storage and multiplication functions. The same memory cells that store weight data are used to perform multiplication operations when appropriate voltages are applied. This multi-functionality approach increases computational efficiency without proportionally increasing device complexity, as the hardware structure remains fundamentally the same while gaining additional processing capability.
4Productivity
If matrix vector multiplication is performed in the memory array, then bandwidth limitations are addressed, but manufacturing precision requirements increase
Solution Approach 1:
The patent utilizes voltage parameter changes to control the operation mode of the memory cell array. By applying different voltage levels to word lines and bit lines, the same memory cells can perform either storage or multiplication operations. This parameter-based control approach enables bandwidth improvement through in-memory computation without requiring additional hardware precision, as the existing memory cells are repurposed through voltage control.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances computational efficiency and reduces power consumption by performing matrix vector multiplication directly in the memory array, thereby addressing bandwidth limitations and power inefficiencies.
Implementation Method 1
A bitline is used to access a set of multi-pillar memory cells. The bitline accumulates output currents from the memory cells to sum a column of weights multiplied by a column of inputs with replacement.
Data Source
AI summary
Systems, methods, and apparatus for memory devices that apply a fixed gate bias to memory cells during programming and multiplication. In one approach, a fixed voltage bias is applied to gates of the memory cells when the cells are programmed. Output currents from the memory cells are measured to control the extent of the programming. A target output current is used for determining when to end programming and corresponds to a weight to be stored. After programming, inputs are applied to the memory cells, and output currents from the memory cells are accumulated to perform matrix vector multiplication.