Far Memory Telemetry for Hot Page Migration and Latency Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of far memory in System on a Chip (SoC) increases memory access latency due to its slower speed and greater distance from the SoC, while near memory is limited by physical size and more expensive.
Innovation Solution
Implementing telemetry logic to monitor page access requests and move frequently accessed pages from far memory to near memory using a telemetry system that tracks page access patterns and manages hot and cold pages, reducing latency by transferring hot pages to near memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If far memory is used to increase storage capacity, then storage space is improved, but memory access latency increases
Solution Approach 1:
The memory system is segmented into two distinct parts: near memory (within SoC) for frequently accessed data and far memory (external) for less frequently accessed data. This segmentation allows the system to optimize for both fast access (near memory) and high capacity (far memory) simultaneously by dividing the memory hierarchy based on access patterns.
Solution Approach 2:
The system dynamically manages memory pages by monitoring access patterns and automatically migrating pages between near and far memory based on their access frequency. This dynamic behavior allows the system to adapt to changing workloads and optimize the balance between storage capacity and access latency in real-time.
2Speed
If near memory is used to reduce memory access latency, then memory access speed is improved, but storage capacity is limited
Solution Approach 1:
The memory system is segmented into two distinct parts: near memory (within SoC) for frequently accessed data and far memory (external) for less frequently accessed data. This segmentation allows the system to optimize for both fast access (near memory) and high capacity (far memory) simultaneously by dividing the memory hierarchy based on access patterns.
Solution Approach 2:
The system creates a copy of frequently accessed data pages by migrating them from far memory to near memory. This copying allows the original data to remain in far memory while a duplicate exists in near memory for fast access, effectively increasing the usable capacity of fast memory through replication of hot data.
3Loss of time
If pages are moved from far memory to near memory, then memory access latency is reduced, but device complexity increases
Solution Approach 1:
The memory management system performs self-service by automatically monitoring access patterns and migrating pages between near and far memory without requiring manual intervention. The telemetry logic continuously tracks page access counts and autonomously makes migration decisions, reducing the need for complex external control mechanisms.
Solution Approach 2:
The system implements feedback mechanisms where telemetry logic monitors page access patterns and uses this information to drive memory migration decisions. The access count records provide feedback about which pages are most frequently accessed, enabling the system to automatically adjust memory placement based on real-time usage data.
Data Source
AI summary
Aspects of the disclosed technology include techniques and mechanisms for using far memory telemetry for hot and cold page management. A processor within a telemetry system parses page access requests transmitted from a central processing unit (CPU) to a computing system. The processor parses the request to determine whether a directory contains a record for the requested page. Based on determining the directory does not store a record for the requested page and based on determining the directory is not at capacity, the processor generates a record and stores the record in the directory. The processor transmits a signal to the computing system to determine whether to perform one or more actions on the pages identified in the directory, such as move a page from a far memory device to a near memory device based on directory data indicating a number of access requests associated with each page.