3D Memory Array In-Memory Multiply-Accumulate
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI hardware solutions for image recognition, such as Convolutional Neural Networks, face challenges with high power consumption and latency due to the large amount of data required for multiply-accumulate (MAC) computations, especially with multi-bit weights.
Innovation Solution
A 3D memory array device that performs MAC operations directly within the memory array, utilizing a 3D memory array with CMOS switches and source line sensing amplifiers to minimize data movement, enabling in-memory computing (IMC) or processor-in-memory (PIM) operations and achieving higher resolution analog weights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If MAC computations are performed using conventional AI hardware solutions with large amounts of multi-bit weights, then computation capability is improved, but power consumption increases and latency increases due to large data transmission requirements
Solution Approach 1:
The patent merges the memory array with the MAC computation circuitry by integrating sense amplifiers that directly perform multiply-accumulate operations on data retrieved from the memory array. This combination eliminates the need for separate data movement between memory and processing units, thereby reducing power consumption while maintaining MAC computation capability.
Solution Approach 2:
The patent transitions from conventional 2D memory architecture to a 3D stacked memory array architecture. This dimensional change enables higher density storage closer to the computation units, reducing the physical distance for data transmission and thereby lowering power consumption and latency for MAC operations.
2Productivity
If MAC computations are performed using conventional AI hardware solutions with large amounts of multi-bit weights, then computation capability is improved, but latency increases due to large data transmission requirements
Solution Approach 1:
The patent merges the memory array with the MAC computation circuitry by integrating sense amplifiers that directly perform multiply-accumulate operations on data retrieved from the memory array. This combination eliminates the need for separate data movement between memory and processing units, thereby reducing latency while maintaining MAC computation capability.
Solution Approach 2:
The patent transitions from conventional 2D memory architecture to a 3D stacked memory array architecture. This dimensional change enables higher density storage closer to the computation units, reducing the physical distance for data transmission and thereby lowering latency for MAC operations.
3Use of energy by moving object
If data movement is reduced by performing MAC operations within the memory array, then power consumption is reduced and latency is reduced, but device complexity increases
Solution Approach 1:
The sense amplifiers in the patent are designed to perform multiple functions: traditional data sensing, MAC computation, and result storage. This multi-functionality reduces the need for separate dedicated circuits, thereby limiting the increase in device complexity while achieving power reduction through in-memory MAC operations.
Solution Approach 2:
The memory array structure is designed to perform MAC operations using its own internal resources (sense amplifiers, bit lines, word lines) without requiring external processing units. This self-service approach enables the memory to compute MAC operations autonomously, reducing power consumption while the complexity is managed through efficient utilization of existing memory components.
Data Source
AI summary
A 3D memory array device includes blocks, bit lines, word lines, source lines (SL), complementary metal oxide semiconductors (COMS), and SL sensing amplifiers (SA). Each block includes NAND strings, and each memory cell in the NAND strings stores one or more weights. The bit lines are respectively coupled as signal inputs to string select lines in all blocks. The word lines are respectively coupled to the memory cells, and the word lines in the same layer are as a convolution layer to perform a convolution operation on the inputted signal. Different SL are coupled to all ground select lines in different blocks to independently collect a total current of the NAND strings in each block. The CMOS are disposed under the blocks and coupled to each source line for transferring the total current to each SL SA, and a multiply-accumulate result of each block is outputted via each SL SA.