Network Adapter Direct Virtual Address Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network adapters in clusters face high overhead due to the need for OS involvement in pinning and managing page registrations for RDMA and zero copy messaging, which limits the performance benefits of these operations and adds complexity, especially in large-scale systems.
Innovation Solution
The solution involves using a context definition table to associate translation requests with independent address spaces, allowing network adapters to perform address translations using CPU page tables and synchronizing with the OS to manage page translations efficiently, enabling direct use of CPU virtual addresses without pinning pages in system memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If network adapters use RDMA and zero copy messaging with pinned memory, then data transfer performance is improved, but system complexity and overhead increase due to OS involvement in pinning and registration
Solution Approach 1:
The patent extracts the memory pinning and registration functions from the operating system and transfers them to the network adapter. The network adapter now directly manages physical address mappings without requiring OS intervention, eliminating the overhead of pinning and registration while maintaining high-performance data transfer
Solution Approach 2:
The network adapter performs self-service by autonomously managing its own address space mappings. It directly uses CPU virtual addresses and performs address translations using the CPU's page tables without external assistance, reducing system complexity and improving efficiency
2Ease of operation
If network adapters require OS involvement for pinning and page registration, then memory management is simplified, but performance benefits of RDMA and zero copy messaging are limited
Solution Approach 1:
The patent introduces a context definition table as an intermediary data structure that enables the network adapter to access and manage page table information directly. This table stores mappings between virtual addresses and physical addresses, allowing the network adapter to perform address translations without OS involvement while maintaining correct memory management
Solution Approach 2:
The patent segments the address translation process into context-specific entries in the context definition table. Each context represents a separate address space with its own page table mappings, allowing the network adapter to efficiently manage multiple address spaces independently without requiring centralized OS control
3Productivity
If network adapters use CPU virtual addresses directly, then overhead of registration and pinning is reduced, but address translation synchronization with OS becomes necessary
Solution Approach 1:
The patent implements a feedback mechanism where the network adapter monitors OS page table updates and adjusts its address translations accordingly. When the OS modifies page tables, the network adapter receives notification and updates its context definition table to maintain consistency, ensuring reliable address translation while using CPU virtual addresses directly
Data Source
AI summary
A system comprising a compute node and coupled network adapter (NA) that allows the NA to directly use CPU virtual addresses without pinning pages in system memory. The NA performs memory accesses in response to requests from various sources. Each request source is assigned to context. Each context has a descriptor that controls the address translation performed by the NA. When the CPU wants to update translation information it sends a synchronization request to the NA that causes the NA to stop fetching a category of requests associated with the information update. The category may be requests associated with a context or a page address. Once the NA determines that all the fetched requests in the category have completed it notifies the CPU and the CPU performs the information update. Once the update is complete, the CPU clears the synchronization request and the NA starts fetching requests in the category.