Stacked DRAM-SRAM Memory for Weight Reuse in AI Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory systems for artificial neural networks, particularly those using DRAM and SRAM, face inefficiencies in energy consumption and storage capacity due to the need for constant refreshing of memory cells when reading weights and limited space for weight storage.
Innovation Solution
A hybrid memory architecture that stacks DRAM and SRAM devices, allowing for multiple reads of memory cells without refreshing and combining their advantages to enhance storage capacity and energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If DRAM is used for weight storage in neural networks, then storage capacity is improved, but energy consumption increases due to constant refreshing requirements
Solution Approach 1:
The patent divides the memory system into two distinct segments: DRAM for bulk weight storage and SRAM for active weight storage. This segmentation allows each memory type to operate in its optimal mode - DRAM provides high-capacity storage without refresh overhead during active computation, while SRAM provides fast, refresh-free access to the subset of weights currently being used.
Solution Approach 2:
The patent introduces an intermediary mechanism (weight buffer and control logic) that mediates between DRAM and SRAM. This intermediary system manages the transfer of weights from DRAM to SRAM, enabling the main DRAM storage to be read without continuous refreshing while maintaining fast access through the SRAM buffer.
2Use of energy by moving object
If SRAM is used for weight storage, then energy efficiency is improved by eliminating refresh operations, but storage capacity is reduced
Solution Approach 1:
The patent merges DRAM and SRAM into a hybrid memory system where both memory types work together. The combination leverages the high capacity of DRAM and the high energy efficiency of SRAM, achieving overall system performance that exceeds what either memory type could provide alone.
Solution Approach 2:
The patent adds a temporal dimension to the memory architecture by implementing a time-multiplexed weight storage strategy._weights are loaded from DRAM to SRAM in advance of computation, allowing the SRAM to be refreshed less frequently or not at all during the computation window, while DRAM handles the bulk storage needs.
3Productivity
If weights are read from memory multiple times during convolution operations, then computational throughput is improved, but power dissipation increases due to repeated memory access
Solution Approach 1:
The patent implements preliminary action by pre-loading weights from DRAM into SRAM before the convolution computation begins. This preliminary transfer enables multiple reads of the same weights during computation without incurring additional DRAM access power costs, as the weights remain in the faster, more energy-efficient SRAM throughout the computation sequence.
Data Source
AI summary
A stacked hybrid memory architecture includes a dynamic random-access memory (DRAM) device. The DRAM device stores a plurality of weights associated with an artificial neural network. The stacked hybrid memory architecture also includes a static random-access memory (SRAM) device bonded to the DRAM device. The SRAM device receives, from the DRAM device through a plurality of through silicon vias (TSVs), the plurality of weights associated with the artificial neural network. The SRAM device also performs a plurality of operations utilizing the plurality of weights. The stacked hybrid memory architecture also includes logic configured to perform a summation operation on a result of the plurality of operations.