PCIe Far Memory Tiering via DMA Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory sub-systems face performance deficiencies due to slower access times in main memory, leading to increased latencies and the need for cache memory to reduce costs while maintaining performance, which results in overhead from compression/decompression and page faults, and bandwidth limitations from DDR buses.
Innovation Solution
Implementing memory tiering using PCIe connected far memory, where a far memory component, such as a cross-point array of non-volatile memory, is accessed via a PCIe bus with a DMA engine for asynchronous data migration between near and far memory, eliminating the need for compression and decompression and allowing direct addressability without page faults, and enabling native virtualization for efficient resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If cache memory is introduced to reduce access latency, then performance is improved, but device complexity and overhead from compression/decompression and page faults increase
Solution Approach 1:
The memory system is segmented into multiple tiers: L1 cache, L2 cache, and persistent memory. Each tier serves a specific function with different performance characteristics, allowing the system to balance speed and complexity by using segmentation rather than a single monolithic memory structure.
Solution Approach 2:
The memory controller acts as an intermediary that manages the complexity of data migration between memory tiers. It handles compression/decompression operations and page fault management automatically, shielding the host system from these complexities while maintaining high performance.
2Quantity of substance
If DDR bus is used for memory communication, then bandwidth is limited, but using faster buses increases device complexity and cost
Solution Approach 1:
The system transitions from a single-dimensional DDR bus to a multi-dimensional memory architecture where data can be accessed through multiple paths: L1 cache, L2 cache, and persistent memory via PCIe. This dimensional expansion allows the system to achieve high bandwidth without requiring a single complex high-speed bus.
Solution Approach 2:
The memory communication path is segmented into multiple interfaces with different bandwidth characteristics. The L1 and L2 caches use high-speed internal buses for low-latency access, while persistent memory uses PCIe for durable storage, allowing each segment to be optimized independently.
3Quantity of substance
If compression is used to reduce data size for cache storage, then memory utilization is improved, but overhead from compression/decompression operations increases
Solution Approach 1:
The system applies compression selectively only to data that needs to be cached, rather than compressing all data. The memory controller determines which data segments benefit from compression based on access patterns and cache pressure, applying compression only when necessary to improve utilization without significantly impacting throughput.
4Ease of operation
If page faults are handled traditionally, then memory management is simplified, but performance deteriorates due to increased latency
Solution Approach 1:
The system performs preliminary actions by pre-loading data into the L1 cache before actual access is needed. The memory controller proactively migrates data between memory tiers and handles page faults in advance, reducing the latency impact when page faults do occur by having data ready beforehand.
Data Source
AI summary
A processing device in a host system monitors a data temperature of a plurality of memory pages stored in a host-addressable region of a cache memory component operatively coupled with the host system. The processing device determines that a first memory page of the plurality of memory pages satisfies a first threshold criterion pertaining to the data temperature of the first memory page and sends a first migration command indicating the first memory page to a direct memory access (DMA) engine executing on a memory-mapped storage component operatively coupled with the cache memory component and with the memory-mapped storage component via a peripheral component interconnect express (PCIe) bus. The first migration command causes the DMA engine to initiate a first DMA transfer of the first memory page from the cache memory component to a host-addressable region of the memory-mapped storage component.