GEMM Dataflow Accelerator with Smart 3D Stacking DRAM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning technologies face challenges in efficiently performing floating point operations for tensor computation due to the 'memory wall' issue, where computation performance outpaces memory bandwidth, and existing accelerators suffer from energy inefficiencies, area overhead, and inadequate memory capacity, making them unsuitable for complex training tasks.
Innovation Solution
A GEMM dataflow accelerator with a smart 3D stacking DRAM architecture that transforms floating point scalar-vector multiplications into concurrent lookup operations using a DRAM-based lookup table, reducing area consumption and latency, and employs a systolic-array dataflow architecture to scale tensor computations across multiple banks, ensuring high bandwidth and efficient data reuse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compute-centric accelerators are used for tensor computation, then computation performance increases, but energy consumption increases significantly due to the memory wall issue and off-chip data movement
Solution Approach 1:
The patent merges computation and memory functions by implementing processing elements directly within the DRAM memory array. Multiply-accumulate operations are performed using memory cell conductance values and readout circuitry, eliminating the need for separate compute units and reducing data movement between memory and compute units, thereby reducing energy consumption while maintaining computation performance
Solution Approach 2:
The patent introduces DRAM-based lookup tables as an intermediary structure to enable efficient tensor computation. The lookup tables store pre-computed values in memory cells, allowing rapid retrieval and computation without requiring complex arithmetic logic units, thus reducing both energy consumption and area overhead while maintaining high computation performance
2Productivity
If floating point units are added to satisfy computational demands, then computation capability increases, but area overhead becomes significant and unacceptable in a DRAM die
Solution Approach 1:
The patent replaces traditional mechanical/computational multiplication units with memory-based lookup operations. Instead of using floating point units that require significant area, the system uses DRAM memory cells to store and retrieve computational values, with readout circuitry performing the multiplication function, thereby dramatically reducing area overhead while maintaining computation capability
Solution Approach 2:
The patent makes the DRAM memory array multi-functional by enabling it to serve both as storage and as the computational substrate. The same memory cells that store data are used to perform multiply-accumulate operations through conductance-based computation, eliminating the need for separate floating point units and reducing area overhead while maintaining full computation capability
3Quantity of substance
If non-volatile memory based accelerators are used, then memory capacity increases, but write endurance decreases and write latency increases, making them unsuitable for write-intensive training tasks
Solution Approach 1:
The patent changes the operational parameters of DRAM to enable persistent storage functionality. By utilizing the inherent charge retention properties of DRAM cells and implementing refresh control logic, the system provides sufficient on-chip memory capacity for training tasks while maintaining the high write endurance and low latency characteristics of volatile memory, avoiding the limitations of non-volatile memory
4Speed
If SRAM-based accelerators are used, then computation speed increases, but on-chip memory capacity is insufficient to store all model parameters and intermediate results
Solution Approach 1:
The patent applies local quality by using high-speed DRAM readout circuitry in the memory array to provide fast access to computational data. The memory array maintains its native high-speed access characteristics while providing large capacity, and computation is performed locally within the memory array using the readout circuitry, thereby achieving both high computation speed and sufficient memory capacity
Data Source
AI summary
A general matrix-matrix multiplication (GEMM) dataflow accelerator circuit is disclosed that includes a smart 3D stacking DRAM architecture. The accelerator circuit includes a memory bank, a peripheral lookup table stored in the memory bank, and a first vector buffer to store a first vector that is used as a row address into the lookup table. The circuit includes a second vector buffer to store a second vector that is used as a column address into the lookup table, and lookup table buffers to receive and store lookup table entries from the lookup table. The circuit further includes adders to sum the first product and a second product, and an output buffer to store the sum. The lookup table buffers determine a product of the first vector and the second vector without performing a multiply operation. The embodiments include a hierarchical lookup architecture to reduce latency. Accumulation results are propagated in a systolic manner.