Far Memory Telemetry for Low-Latency Hot Page Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of far memory in System on a Chip (SoC) systems increases memory access latency due to its slower speed and greater distance from the SoC, while near memory is limited by physical size and more expensive.
Innovation Solution
Implementing telemetry logic to manage pages by moving frequently accessed pages from far memory to near memory using telemetry processors that monitor and analyze page access requests, updating directories with access counts, and moving pages based on access thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If far memory is used to increase storage capacity, then storage capacity is improved, but memory access latency increases
Solution Approach 1:
The memory system is segmented into near memory (integrated with SoC) and far memory (external to SoC). The telemetry processor segments page access requests into hot pages (frequently accessed) and cold pages (infrequently accessed), storing hot pages in near memory and cold pages in far memory, thus resolving the contradiction between storage capacity and access latency
Solution Approach 2:
Different memory locations are assigned different qualities: near memory provides fast access for frequently used data, while far memory provides large capacity for less frequently used data. The telemetry processor dynamically determines which pages receive the 'hot page' quality treatment by monitoring access patterns, allowing the system to optimize both capacity and latency
2Speed
If near memory is used to reduce access latency, then memory access speed is improved, but physical size and cost increase
Solution Approach 1:
The system dynamically adjusts the contents of near memory based on real-time access patterns monitored by the telemetry processor. Pages are dynamically moved between near and far memory based on their access frequency, allowing near memory to maintain high speed performance for active data without being permanently allocated for all potentially needed data
Solution Approach 2:
The telemetry processor performs preliminary monitoring of page access patterns to identify hot pages before they are needed frequently. By proactively moving pages to near memory based on predicted access patterns, the system prepares the fast memory in advance, reducing latency before it becomes a problem
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Aspects of the disclosed technology include techniques and mechanisms for using far memory telemetry for hot and cold page management. A processor within a telemetry system parses page access requests transmitted from a central processing unit (CPU) to a computing system. The processor parses the request to determine whether a directory contains a record for the requested page. Based on determining the directory does not store a record for the requested page and based on determining the directory is not at capacity, the processor generates a record and stores the record in the directory. The processor transmits a signal to the computing system to determine whether to perform one or more actions on the pages identified in the directory, such as move a page from a far memory device to a near memory device based on directory data indicating a number of access requests associated with each page.