Far-Memory Page Access Probes for Low-Overhead Hot/Cold Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Disaggregated computing architectures experience increased memory latency due to the overhead of conventional hot/cold page detection methods, which are resource-intensive and inefficient in managing memory access in multi-tiered systems.
Innovation Solution
A novel data structure and page access probe mechanism that reduces CPU cycles by selectively sampling and filtering memory pages, using kernel hooks and callbacks to update page meta data asynchronously, allowing for efficient hot/cold page detection and movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional hot/cold page detection methods are used in disaggregated computing architectures, then memory access patterns can be identified, but CPU overhead increases significantly and memory latency increases
Solution Approach 1:
The patent segments the memory system into multiple tiers (fast local memory and slower disaggregated memory) and applies different detection strategies to each tier. The probe mechanism is segmented into multiple probes that can operate independently on different memory pages, allowing parallel detection that reduces overall latency while maintaining detection accuracy.
Solution Approach 2:
The patent implements periodic sampling of memory access patterns through the probe mechanism, which periodically checks memory pages to detect hot/cold patterns. This periodic action allows the system to balance detection accuracy with reduced CPU overhead by not continuously monitoring all memory pages, thus lowering memory latency while maintaining effective hot/cold page identification.
2Productivity
If conventional hot/cold page detection methods are used, then memory management can be performed, but CPU cycles are excessively consumed
Solution Approach 1:
The probe mechanism is designed to operate with minimal CPU intervention, utilizing hardware-supported probe operations that can execute with reduced CPU overhead. The system enables self-service memory management where the probe mechanism autonomously collects memory access pattern data, reducing the CPU cycles required for memory management while maintaining productivity.
Solution Approach 2:
The patent replaces traditional software-based memory management mechanisms with a hardware-supported probe mechanism. This substitution reduces CPU cycle consumption by offloading detection operations to specialized hardware probes that can operate more efficiently than general-purpose CPU instructions, thereby improving memory management efficiency while reducing energy consumption.
3Use of energy by moving object
If selective sampling and filtering of memory pages is implemented, then CPU overhead is reduced, but detection complexity increases
Solution Approach 1:
The detection mechanism is segmented into multiple specialized probes, each responsible for specific memory pages or regions. This segmentation reduces CPU overhead by distributing detection tasks across multiple independent probes that can operate in parallel, while the modular structure manages complexity through clear separation of detection responsibilities.
Solution Approach 2:
The patent implements selective sampling that performs partial detection on only the most critical or frequently accessed memory pages rather than exhaustive detection of all pages. This partial action reduces CPU overhead by focusing resources on high-priority pages, while the filtering mechanism manages complexity by selectively applying detection only where needed.
Data Source
AI summary
Methods and apparatus to minimize hot/cold page detection overhead on running workloads. A page meta data structure is populated with meta data associated with memory pages in one or more far memory tier. In conjunction with one or more processes accessing memory pages to perform workloads, the page meta data structure is updated to reflect accesses to the memory pages. The page meta data is used to determine which pages are “hot” pages and which pages are “cold” pages, wherein hot pages are memory pages with relatively higher access frequencies and cold pages are memory pages with relatively lower access frequencies. Variations on the approach including filtering meta data updates on pages in memory regions of interest and applying a filter(s) to trigger meta data updates based on (a) condition(s). A callback function may also be triggered to be executed synchronously with memory page accesses.