CXL Memory Page Migration for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CXL memory expansion methods in computer architectures face challenges in load balancing across memory components, leading to significant latency swings during data transfers, especially when frequently accessed data straddles pages between local DDR and CXL memory tiers.
Innovation Solution
A method and system that track frequently accessed pages (hot pages) in external CXL memory and move them to a higher performance local DDR layer using a Direct Memory Access (DMA) engine with a page queue interface, thereby minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If CXL memory expansion is used to add memory capacity, then memory capacity is increased, but data transfer latency increases significantly
Solution Approach 1:
The patent applies local quality by differentiating between hot pages and cold pages, and between local DDR memory and remote CXL memory. Frequently accessed pages are kept in local DDR memory with fast access characteristics, while less frequently accessed pages are stored in CXL memory with larger capacity but slower access. This spatial differentiation of data based on access patterns resolves the contradiction by ensuring that capacity needs are met through CXL while latency-critical operations use local memory.
Solution Approach 2:
The patent implements preliminary action through page migration mechanisms that proactively move hot pages from CXL memory to local DDR memory before they are needed. The system monitors access patterns and anticipates future access needs, pre-positioning data in the optimal location (local DDR) to minimize latency when access occurs. This proactive approach resolves the contradiction by preparing the system in advance rather than reacting to latency issues when they occur.
2Quantity of substance
If data structures straddle pages between local DDR and CXL memory, then memory capacity utilization is improved, but load balancing deteriorates and latency increases
Solution Approach 1:
The patent applies local quality by ensuring that complete data structures are kept together in the same memory location (either both in local DDR or both in CXL). The system monitors access patterns at the data structure level rather than individual page level, and migrates entire data structures to maintain locality. This prevents the load balancing deterioration that would occur if data structures were fragmented across memory tiers with different access characteristics.
Solution Approach 2:
The patent merges related pages that form complete data structures into the same memory tier. When a data structure is identified as hot, all pages comprising that structure are migrated together to local DDR memory, even if some pages were previously in CXL. This combining approach ensures that data structures are not straddled across memory tiers, maintaining load balance and preventing the latency swings that would result from partial data structure access.
3Power
If CXL memory is used for memory expansion, then memory bandwidth is increased, but latency swings between 40-60 ns and 400-600 ns occur
Solution Approach 1:
The patent applies local quality by creating a two-tier memory system where latency-sensitive operations access local DDR memory (40-60 ns) while capacity-sensitive operations access CXL memory (400-600 ns). The system dynamically classifies pages as hot or cold and positions them in appropriate memory tiers, ensuring that the bandwidth of CXL is utilized for capacity expansion while the latency consistency of local DDR is maintained for active data. This resolves the latency swing contradiction by ensuring that accessed data resides in the appropriate memory tier.
Data Source
AI summary
A system for tracking and moving pages within a memory hierarchy is disclosed. In some embodiments, the system comprises a memory hierarchy having low-tier memory and high-tier memory. The system comprises an input/output (I/O) port configured to map into the low-tier memory. The system comprises a central processing unit (CPU) associated with the high-tier memory and configured to make one or more page requests for accessing a page stored in the low-tier memory via the I/O port. The system also comprises a page tracker configured to determine a count of the one or more page requests. The system further comprises a data movement engine configured to move content of the page from the low-tier memory to the high-tier memory when the count of the one or more page requests exceeds a predetermined threshold.