Memory Subsystem Address Translation Cache for Lower PCIe Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The need to continually request a translation agent (TA) for address translations in memory sub-systems, particularly in systems with multiple non-volatile memory express (NVMe) devices, leads to performance bottlenecks in terms of speed, latency, and quality-of-service, due to increased I/O traffic and address translation requests across the PCIe bus.
Innovation Solution
Implementing an address translation cache (ATC) within the memory sub-system controller to store address translations for future access, reducing the need to repeatedly request TA for translations, especially for sequential memory command queues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If address translation requests are sent to the translation agent for every memory command, then address translation accuracy is maintained, but I/O traffic and latency increase
Solution Approach 1:
The patent pre-fetches and caches address translations for sequential memory commands before they are actually executed. The translation agent translates addresses in advance and stores them in a cache, so when sequential commands arrive, their translations are already available, eliminating the need for real-time translation requests and reducing latency.
Solution Approach 2:
The system performs address translation proactively for the next expected command in sequence before the current command completes execution. This lookahead approach ensures that translations are ready when needed, maintaining accuracy while reducing the perceived latency for command execution.
2Loss of information
If address translation requests are sent frequently to the translation agent, then address translation completeness is ensured, but I/O traffic across the PCIe bus increases
Solution Approach 1:
The system anticipates sequential memory commands and pre-translates their addresses before the commands are executed. By caching these translations in advance, the system ensures that all necessary address translations are complete without requiring frequent real-time requests to the translation agent, thereby reducing PCIe bus traffic.
Solution Approach 2:
The patent creates copies of address translation entries and stores them in a cache within the memory sub-system controller. Instead of repeatedly querying the translation agent for the same translations, the system uses these cached copies, ensuring translation completeness while minimizing I/O traffic across the PCIe bus.
3Measurement precision
If the address translation cache is frequently invalidated to maintain accuracy, then translation correctness is preserved, but cache hit rate and throughput decrease
Solution Approach 1:
The system pre-translates addresses for sequential commands and caches them with validity tracking. By proactively managing the cache with pre-fetched translations and implementing intelligent invalidation based on actual command execution patterns, the system maintains translation correctness while maximizing cache hit rates and throughput.
4Measurement precision
If the memory sub-system waits for address translation from the host system, then translation accuracy is ensured, but processing speed decreases
Solution Approach 1:
The memory sub-system proactively requests and caches address translations from the translation agent before the host system actually needs them for command execution. This pre-fetching approach allows the sub-system to maintain accurate translations while processing commands at full speed without waiting for host system translation requests.
Solution Approach 2:
The memory sub-system takes initiative to manage its own address translation needs by maintaining a local cache and proactively requesting translations from the translation agent. Instead of passively waiting for the host system to provide translations, the sub-system serves its own translation requirements, significantly improving processing speed while maintaining accuracy.
Data Source
AI summary
A system includes host interface circuitry to interact with a host system and that includes an address translation circuit, which includes request staging queues to buffer the address translation requests, each includes a virtual address and received from a host interface circuit. Pending response queues buffer respective address translation requests that are waiting for an address translation from the host system while maintaining an order as received within the request stage queues. Reordering buffers reorder address translations, which are to be supplied to the host interface circuits, according to the order maintained within the pending response queues, each address translation includes a physical address mapped to the virtual address of a corresponding address translation request. A cache stores multiple of the address translations, associated with the address translation requests, for future access by the host interface circuits.