Memory-Sampling Page Cache Migration for Latency-Capacity Trade-offs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Determining how to allocate memory pages across multiple levels of memory in computing systems is challenging due to differences in performance, cost, and resiliency, particularly in systems with high-bandwidth stacked memories and traditional off-package memory, where latency and bandwidth requirements need to be balanced with capacity considerations.
Innovation Solution
A system that records metadata from memory access instructions, predicts which pages will be accessed in subsequent intervals, and migrates pages between memory levels based on access frequencies, using a multi-level memory hierarchy with a fast, limited first level and a larger, slower second level, while optionally adjusting sampling intervals based on memory accesses or page faults.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If memory pages are allocated to faster memory level, then latency is reduced and bandwidth is increased, but capacity is limited
Solution Approach 1:
The memory system is segmented into multiple levels (fast stacked memory and slower off-chip memory), with the patent allocating frequently accessed pages to the fast level and less frequently accessed pages to the slower level, thereby resolving the contradiction between speed and capacity through hierarchical organization
Solution Approach 2:
The patent implements dynamic page migration between memory levels based on access patterns detected through sampling. Pages are moved from slow to fast memory when access frequency increases, and vice versa, making the memory allocation dynamic rather than static to optimize both speed and effective capacity
2Productivity
If more memory pages are placed in fast memory level, then bandwidth and latency improve, but the limited capacity of fast memory is exceeded
Solution Approach 1:
The patent samples a portion of memory access instructions rather than tracking all accesses, and migrates only the most frequently accessed pages to fast memory. This partial action approach achieves sufficient bandwidth improvement without exceeding fast memory capacity
Solution Approach 2:
The system changes the parameter of page location between memory levels based on access frequency parameters. By adjusting which pages reside in fast memory versus slow memory, the system optimizes bandwidth utilization within the constrained capacity of fast memory
3Measurement precision
If memory access instructions are sampled and analyzed, then page allocation accuracy improves, but system overhead increases
Solution Approach 1:
The patent samples a subset of memory access instructions rather than analyzing every single access. This partial sampling approach provides sufficient accuracy for effective page migration while keeping the overhead manageable and not excessively burdening the system
Solution Approach 2:
The memory management system performs self-service by automatically sampling its own access patterns and making migration decisions without requiring external intervention or complex external control mechanisms, thereby reducing overall system complexity
Data Source
AI summary
Systems, apparatuses, and methods for implementing a memory sampling based migrating page cache are disclosed. In one embodiment, a system includes one or more processors and a multi-level memory hierarchy. The system is configured to record metadata associated with a portion of memory access instructions executed by one or more processors in one or more sampling intervals. The system generates predictions on which memory pages will be accessed in a subsequent sampling interval based on the recorded metadata. The system migrates one or more memory pages to a first memory level from a second memory level responsive to predicting that the one or more memory pages will be accessed in the subsequent sampling interval. The system also adjusts a duration of the sampling interval based on the number of memory accesses or a number of page faults per interval.