NVLink Memory Bridging for Host Address Space Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current NVLink-based systems face challenges in enabling NVLink-connected devices to access host memory resources due to the separation between device and host physical address spaces, limiting efficient memory sharing and utilization in memory-intensive workloads.
Innovation Solution
Implementing Resource Provisioning Units (RPUs) that translate physical addresses associated with NVLink-based protocols to physical addresses within the host's physical address space, allowing NVLink-connected devices to access host memory as extended resources, supporting memory disaggregation and pooling configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If NVLink-connected devices use separate physical address spaces, then device memory capacity is limited to local device memory, but address space separation creates barriers to efficient memory sharing
Solution Approach 1:
The patent introduces a bridge device as an intermediary between NVLink-connected devices and host memory subsystems. This bridge performs address translation between device physical addresses and host physical addresses, enabling NVLink devices to access host memory resources while maintaining the high-bandwidth NVLink interface. The bridge acts as a mediator that resolves the address space separation issue without sacrificing memory sharing efficiency
Solution Approach 2:
The patent extends the addressable memory space by adding another dimension - host memory resources accessible through the bridge. Instead of being limited to the one-dimensional local device memory space, NVLink-connected devices can now access a multi-dimensional memory space that includes both local device memory and remote host memory, effectively increasing the available memory capacity
2Productivity
If host memory is made accessible to NVLink devices, then memory resource utilization improves, but address translation complexity increases
Solution Approach 1:
The bridge device serves as a specialized intermediary that handles all address translation operations, isolating the translation complexity from both the NVLink devices and the host processor. This allows the devices to maintain simple address spaces while the bridge manages the translation complexity, improving memory resource utilization without significantly increasing device complexity
Solution Approach 2:
The patent segments the memory access function into distinct components: NVLink devices generate memory access requests, the bridge performs address translation and protocol conversion, and the host memory subsystem provides storage. This segmentation allows each component to be optimized independently, improving overall memory resource utilization while managing complexity through functional separation
Data Source
AI summary
Modem datacenters require efficient mechanisms for memory resource sharing across heterogeneous computing environments to support AI workloads, LLM inference, and high-performance computing applications. Some of the disclosed embodiments introduce systems and methods incorporating an RPU that performs address translations between NVLink-based protocols and host physical address spaces, enabling GPUs, accelerators, and other NVLink-capable devices to access host memory resources. The system includes processing cores with MMUs, a coherent interconnect coupling the cores to memory controllers supporting more than 64 GB of memory, and an RPU with an NVLink-based interface. The RPU translates physical addresses associated with the NVLink-based protocol to physical addresses within the host's physical address space, enabling entities to access host memory via the NVLink-based interface. The embodiments optionally support memory disaggregation and pooling configurations, enabling flexible memory architectures and improved resource utilization suitable for GenAI workloads, distributed computing, and next-generation datacenter deployments.