CXL Memory Expansion Cache for Lower-Latency NVM Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for faster data processing in applications like cloud computing and artificial intelligence is hindered by memory capacity and bandwidth gaps, with memory speed and latency not keeping pace with processor advancements, leading to inefficiencies in software-defined memory expansion.
Innovation Solution
A memory expansion device utilizing non-volatile memory as tier 1, optional device DRAM as tier 2, and cache memory as tier 3, coupled via a Computer Express Link (CXL) bus, with control logic optimizing data transfers and caching to minimize latency and maximize bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If memory capacity is increased to keep up with processor speed, then memory capacity increases, but memory latency increases at a rate of about 1.1 times every two years
Solution Approach 1:
The memory system is segmented into multiple tiers with different characteristics: tier 1 uses NVM for high capacity and low cost, tier 2 uses device DRAM for coherent memory with high bandwidth, and tier 3 uses device cache for low latency operations. This segmentation allows each tier to optimize for its specific function while collectively solving the capacity-latency tradeoff.
Solution Approach 2:
The patent introduces a new dimension of memory hierarchy by adding tier 2 (device DRAM) between tier 1 (NVM) and tier 3 (device cache), creating a three-dimensional memory architecture. This additional layer provides both capacity expansion and bandwidth improvement, addressing the limitations of traditional two-tier systems.
2Quantity of substance
If software-defined memory expansion using NVMe SSD is used, then memory capacity expansion is achieved, but performance issues arise including high latency penalty for page faults and poor quality of predictive prefetching
Solution Approach 1:
Device DRAM serves as an intermediary buffer between NVM and device cache, mediating data transfers and reducing the latency penalty for page faults. The device DRAM absorbs the speed mismatch between NVM and cache, providing consistent performance across different workload types.
Solution Approach 2:
The system performs preliminary actions by using device cache to store frequently accessed data before it reaches the NVM tier, and using predictive prefetching algorithms that leverage the device DRAM buffer to anticipate future memory needs, thereby reducing page fault penalties.
3Speed
If higher memory speed is implemented, then memory bandwidth increases, but memory capacity decreases
Solution Approach 1:
Different parts of the memory system have different qualities optimized for their specific functions: device cache provides high speed for small, frequently accessed datasets; device DRAM provides high bandwidth for medium-sized working sets; and NVM provides high capacity for large-scale storage. Each component's local quality matches its operational requirements.
Data Source
AI summary
A memory expansion device operable with a host computer system (host) comprises a non-volatile memory (NVM) subsystem, cache memory, and control logic configurable to receive a submission from the host including a read command and specifying a payload in the NVM subsystem and demand data in the payload. The control logic is configured to request ownership of a set of cache lines corresponding to the payload, to indicate completion of the submission after acquiring ownership of the cache lines, and to load the payload to the cache memory. The set of cache lines correspond to a set of cache lines in a coherent destination memory space accessible by the host. The control logic is further configured to, after indicating completion of the submission and in response to a request from the host to read demand data in the payload, return the demand data after determining that the demand data is in the cache memory.