Memory Tiering With CXL Access Logs to Reduce NUMA Scan Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory tiering systems face challenges with high latency due to non-uniform memory access (NUMA) balancing scans, which cause CPU resource wastage, page faults, and unpredictable access sequences, leading to suboptimal system performance.
Innovation Solution
The proposed systems and methods utilize a compute express link (CXL) device capability hint to avoid NUMA balancing scans by using access logs to determine page promotions and demotions between memory tiers, optimizing memory allocation based on access counters and data structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If NUMA balancing scans are used for memory tiering, then memory allocation can be optimized, but CPU resource wastage and latency increase
Solution Approach 1:
The system performs preliminary actions by maintaining access logs that record memory access patterns before NUMA balancing scans are needed. When a page fault occurs, the system can quickly determine promotion/demotion decisions based on pre-collected access log data, avoiding the need for lengthy scanning operations and reducing latency while still achieving optimal memory allocation.
Solution Approach 2:
The invention extracts only the necessary information (access counters and access sequences) from the access logs when needed, rather than performing comprehensive scans of all memory structures. This selective extraction of relevant data minimizes CPU resource wastage while maintaining effective memory tiering decisions.
2Reliability
If NUMA balancing scans are performed, then memory tiering can be maintained, but CPU resource wastage increases
Solution Approach 1:
The access logs automatically record memory access patterns in the background without requiring active CPU intervention for data collection. The logs self-update with each memory access, allowing the system to maintain reliable memory tiering information without continuous CPU resource consumption during scanning operations.
Solution Approach 2:
Access log data is collected and prepared in advance, so when memory tiering decisions need to be made, the CPU only needs to read pre-processed information rather than performing resource-intensive scans. This preliminary data preparation significantly reduces CPU resource wastage while maintaining tiering reliability.
3Productivity
If traditional memory tiering methods are used, then system performance can be maintained, but access sequencing accuracy decreases
Solution Approach 1:
The system implements feedback mechanisms by continuously monitoring and recording access sequences in the logs. This feedback information about actual access patterns allows the system to make more accurate promotion and demotion decisions, improving both access sequencing accuracy and overall system performance through data-driven memory management.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are systems (300), methods, and apparatuses for systems (300) and methods for reducing latency of memory tiering devices. In one or more examples, the systems (300), devices, and methods include determining a number of pages (335) in a first memory tier (330) satisfies a threshold; based on the number of pages (335) in the first memory tier (330) satisfying the threshold, obtaining from an access log (390) an access counter (394) and a first physical address (392) associated with a page (355) of a second memory tier (360); translating the first physical address (392) to a second physical address (380) associated with a host; and modifying, based on the access counter (394), a counter field of a first data structure (385).