CXL Memory Controller Hot-Address Tracking to Cut Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory systems face challenges in managing access frequencies to non-volatile memory, leading to latency deterioration and inefficient data handling.
Innovation Solution
A memory system that includes a memory controller managing access frequencies by using a histogram to track access patterns, selectively updating and transmitting telemetry information, and optimizing data storage and retrieval processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the memory controller reads and updates access frequency data from the histogram in the volatile memory for every access, then the access frequency tracking is accurate, but the latency increases due to frequent volatile memory accesses
Solution Approach 1:
The system pre-identifies hot addresses (frequently accessed addresses) and stores them in a separate hot address table. When an access occurs to a hot address, the system directly updates the access count in the hot address table without reading from or writing to the histogram in volatile memory, thus eliminating the latency penalty while maintaining accurate tracking.
Solution Approach 2:
The access frequency tracking is segmented into two parts: hot addresses stored in a fast hot address table and cold addresses stored in the histogram in volatile memory. This segmentation allows the system to handle frequent accesses to hot addresses quickly while maintaining comprehensive tracking for all addresses, resolving the contradiction between accuracy and latency.
2Loss of information
If the memory controller transmits telemetry information for all access frequencies, then the host has complete information for optimization, but the data transmission volume and processing overhead increase
Solution Approach 1:
The system extracts only the hot address information and their access frequencies from the complete histogram data and transmits only this subset to the host. This extraction principle allows the host to receive the most relevant information for optimization (hot addresses that need attention) while avoiding the overhead of transmitting complete histogram data, thus improving transmission efficiency without losing critical information.
Solution Approach 2:
The telemetry transmission focuses on local quality by prioritizing hot address information over cold address information. The host receives detailed access frequency data for hot addresses (which are most important for performance optimization) while cold addresses receive less attention or no transmission, optimizing the information quality for the host's needs without unnecessary data overhead.
3Loss of information
If the memory controller uses a large histogram in volatile memory to track all access frequencies, then the access pattern tracking is comprehensive, but the volatile memory usage and access overhead increase
Solution Approach 1:
The access pattern tracking is segmented into hot addresses stored in a compact hot address table in volatile memory and cold addresses tracked in the histogram (which can be in non-volatile memory or less frequently accessed). This segmentation reduces the amount of volatile memory needed for high-speed access while maintaining comprehensive tracking capability for all addresses.
Solution Approach 2:
The system pre-identifies and separates hot addresses into a dedicated table before normal access operations begin. This preliminary action allows the majority of volatile memory to be used for other purposes while hot addresses are tracked in a small, fast table, reducing overall volatile memory requirements without compromising tracking completeness.
Data Source
AI summary
A memory system is configured to access a host in conformity with a CXL™ (Compute Express Link) interface protocol. The memory system includes: a non-volatile memory; a volatile memory; and a memory controller configured to manage a number of accesses to the non-volatile memory by address and store numbers of accesses in the volatile memory as a histogram. The memory controller includes an update unit configured to, in reception of a command from the host: in a case where a condition is satisfied, skip reading of a number of accesses for a command address designated by the command, and in a case where the predetermined condition is not satisfied, read the number of accesses for the command address, and change the number of accesses for the command address stored in the volatile memory.