Remote Memory Prefetch via Virtual Address Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for accessing remote memory, such as RDMA reads and writes, are inefficient for small payloads due to address space changes and high latency, which can significantly impact CPU performance in large systems.
Innovation Solution
A method for prefetching data from remote memory into a cache without changing the address space, involving a compute node interface device that translates virtual addresses to physical addresses and stores data in a cache with corresponding tags, allowing for efficient data retrieval and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If RDMA reads and writes are used to transfer data, then network efficiency is improved, but address space changes occur and latency increases for small payloads
Solution Approach 1:
The system performs preliminary actions by prefetching data from remote memory into the cache before the CPU actually needs it. The interface device monitors CPU access patterns and initiates data transfer in advance, storing data in the cache so that when the CPU requests the data, it can be retrieved quickly without waiting for remote memory access.
Solution Approach 2:
The interface device acts as an intermediary between the CPU and remote memory. It includes a cache that serves as a buffer, translating virtual addresses to physical addresses and managing data transfer between remote memory and CPU memory, thereby reducing the latency impact on the CPU.
2Stability of the object's composition
If CPU loads and stores are used to access remote memory, then address space consistency is maintained, but CPU wait time increases due to high latency
Solution Approach 1:
The interface device performs preliminary data prefetching into the cache before the CPU needs the data. By monitoring virtual address access patterns and proactively transferring data to the cache, the system ensures that when the CPU performs loads or stores, the data is already available locally, eliminating CPU wait time while maintaining address space consistency.
Solution Approach 2:
The interface device autonomously monitors CPU access patterns and automatically prefetches data without CPU intervention. It detects virtual address sequences, translates them to physical addresses, and initiates data transfer to the cache independently, allowing the CPU to continue execution without waiting for remote memory access.
3Productivity
If data is prefetched into cache, then data access efficiency is improved, but address space changes occur requiring translation
Solution Approach 1:
The interface device serves as an intermediary that handles address translation between virtual addresses used by the CPU and physical addresses used for cache storage. It includes address translation logic that converts virtual addresses to physical addresses, managing the complexity of address space mapping while allowing the CPU to work with simple virtual addresses.
Solution Approach 2:
The system creates a copy of the remote memory data in the local cache with associated tag information that includes the physical address. This copying mechanism allows the CPU to access data efficiently from the cache using virtual addresses, while the interface device maintains the mapping between virtual and physical addresses in the cache tags.
Data Source
AI summary
Provided is a method, which may be performed on a computer, for prefetching data over an interface. The method may include receiving a first data prefetch request for first data of a first data size stored at a first physical address corresponding to a first virtual address. The first data prefetch request may include second data specifying the first virtual address and third data specifying the first data size. The first virtual address and the first data size may define a first virtual address range. The method may also include converting the first data prefetch request into a first data retrieval request. To convert the first data prefetch request into a first data retrieval request the first virtual address specified by the second data may be translated into the first physical address. The method may further include issuing the first data retrieval request at the interface, receiving the first data at the interface and storing at least a portion of the received first data in a cache. Storing may include setting each of one or more cache tags associated with the at least a portion of the received first data to correspond to the first physical address.