Address Translation Cache for NVMe Memory Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The continual need for address translations in memory sub-systems, particularly in systems supporting multiple non-volatile memory express (NVMe) devices, leads to performance bottlenecks in terms of speed, latency, and quality-of-service due to increased I/O traffic and the need to move translation requests across the PCIe bus.
Innovation Solution
Implementing an address translation cache (ATC) within the memory sub-system's controller to store address translations for future access, reducing the need for repeated translations by the translation agent (TA) and minimizing I/O traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If address translation requests are processed through the PCIe bus for each memory access, then address translation accuracy is maintained, but system latency increases and I/O traffic grows
Solution Approach 1:
The patent implements an address translation cache that pre-stores address translation mappings before they are needed for memory access. When a memory access request arrives, the controller checks the cache first to see if the translation is already available, eliminating the need for real-time PCIe bus transactions for every access. This preliminary caching of translation data directly reduces system latency while maintaining translation accuracy through validated cache entries.
Solution Approach 2:
The address translation cache acts as an intermediary component between the memory controller and the PCIe bus. Instead of every memory access request directly querying the PCIe bus for address translations, the cache mediates by providing translations from its stored mappings. This intermediary layer filters out redundant PCIe transactions, reducing I/O traffic and latency while preserving the accuracy of address translations through proper cache management.
2Reliability
If address translation requests are sent across the PCIe bus for every memory access, then up-to-date translations are ensured, but I/O traffic increases and throughput decreases
Solution Approach 1:
The controller pre-loads address translation mappings into the cache before memory access operations begin. This preliminary action ensures that when memory access requests arrive in bulk, the translations are already available in the cache, enabling high-speed processing without repeated PCIe bus queries. The cache is updated with current translations through periodic refreshes or event-driven updates, maintaining translation currency while enabling high throughput during active operations.
Solution Approach 2:
The address translation cache enables continuous memory access operations without interruption for translation lookups. Once translations are loaded into the cache, multiple memory access requests can be processed continuously using these cached translations, maintaining high throughput. The system ensures translation currency through continuous cache management activities such as invalidation of stale entries and reloading of updated translations, allowing productive work to continue uninterrupted.
3Adaptability or versatility
If multiple NVMe devices are supported simultaneously, then system versatility is improved, but I/O traffic and translation overhead increase
Solution Approach 1:
The address translation cache is designed as a universal resource that serves multiple NVMe devices simultaneously. The cache stores translation mappings that can be shared across different devices, allowing the same cache structure to handle translation requests from any connected NVMe device. This multi-functional cache reduces the need for separate translation mechanisms for each device, lowering overall I/O traffic overhead while maintaining support for multiple devices through a single unified caching system.
Solution Approach 2:
The patent combines address translation functionality for multiple NVMe devices into a single unified cache structure rather than maintaining separate translation caches for each device. This merging of translation resources allows the controller to serve multiple devices through one cache, reducing redundant translation storage and minimizing I/O traffic. The unified cache manages translations from all devices collectively, optimizing resource utilization and reducing overhead while maintaining full multi-device support.
Data Source
AI summary
A system includes host interface circuits to interact with a host system and an address translation circuit to handle address translation requests to the host system from the host interface circuits. The address translation circuit includes several components as follows. Request staging queues buffer the address translation requests received from a host interface circuit. Pending response queues buffer respective address translation requests, in an order received, that are waiting for an address translation from the host system. Reordering buffers reorder address translations, which are to be supplied to the host interface circuits, according to the order of the received address translation requests maintained within the set of pending response queues. A cache stores a plurality of the address translations, associated with the address translation requests, received from the host system.