Hybrid Memory System with DRAM Cache for Accelerator Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional accelerator platforms face challenges with low memory capacity and high memory bandwidth demands, leading to frequent data migration between external storage devices and accelerators, which decreases application processing performance and increases power consumption.
Innovation Solution
A hybrid memory system is introduced, comprising a resistance-based non-volatile memory, a DRAM used as a cache, a non-volatile memory controller, a memory controller, and a memory channel. This system manages data migration between the DRAM and the non-volatile memory, allowing for parallel data transfers and optimizing cache usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in external storage device, then memory capacity is increased, but data transfer time and power consumption increase due to frequent data migration
Solution Approach 1:
The memory system is segmented into multiple DRAM banks and non-volatile memory banks, allowing data to be distributed across different memory types. This segmentation enables parallel data access and reduces the need for frequent data migration between external storage and accelerator, as data can be kept in the hybrid memory system closer to the processing unit.
Solution Approach 2:
A hybrid memory controller is introduced as an intermediary between the accelerator and the hybrid memory system. This controller manages data placement and migration between DRAM and non-volatile memory, optimizing data access patterns and reducing transfer time by intelligently caching frequently accessed data in DRAM while maintaining large capacity through non-volatile memory.
2Quantity of substance
If data is stored in external storage device, then memory capacity is increased, but power consumption increases due to frequent data migration
Solution Approach 1:
Data is pre-loaded into the hybrid memory system from external storage before processing begins. The hybrid memory controller proactively manages data placement, predicting which data will be needed and loading it into DRAM or non-volatile memory in advance, reducing the need for frequent data migration during processing and thereby lowering power consumption.
Solution Approach 2:
The hybrid memory controller acts as an intermediary that optimizes power consumption by managing data residency in the memory system. It keeps frequently accessed data in the hybrid memory system, reducing migrations to external storage, and intelligently manages the balance between DRAM and non-volatile memory to minimize energy-intensive data transfers.
3Speed
If DRAM is used for high bandwidth memory, then data transfer speed is improved, but memory capacity is reduced
Solution Approach 1:
The system merges DRAM and non-volatile memory into a unified hybrid memory system that appears as a single memory resource to the accelerator. This combination allows the system to leverage the high speed of DRAM for frequently accessed data while using non-volatile memory to provide large capacity, achieving both high data transfer speed and large memory capacity simultaneously.
Solution Approach 2:
Different regions of the hybrid memory system have different characteristics - DRAM banks provide high-speed access for active data, while non-volatile memory banks provide high-capacity storage for less frequently accessed data. The hybrid memory controller dynamically assigns data to appropriate regions based on access patterns, ensuring that speed-critical data resides in DRAM while capacity requirements are met by non-volatile memory.
Data Source
AI summary
An accelerator includes a processor and a hybrid memory system. The hybrid memory system includes a resistance-based non-volatile memory, a DRAM used as a cache of the resistance-based non-volatile memory, a non-volatile memory controller connected to the resistance-based non-volatile memory and configured to control the DRAM and the resistance-based non-volatile memory, a memory controller configured to process a memory request from the processor and control the DRAM, and a memory channel configured to connect the DRAM, the non-volatile memory controller, and the memory controller.