Shared Virtual Memory API for Cross-Node GPU Memory Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face inefficiencies in sharing memory between GPUs across different nodes in a GPU cluster, requiring expensive operations to copy memory and complicating programming efforts.
Innovation Solution
Implementing an application programming interface (API) to allocate, designate, and manage shared virtual memory across multiple processors, utilizing multicast memory handles to facilitate efficient memory sharing and access without direct knowledge of node updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If memory copying operations are used to share data between GPUs on different nodes, then data can be transferred between processors, but the operation becomes expensive and resource-intensive
Solution Approach 1:
The patent introduces shared virtual memory as an intermediary mechanism between GPUs on different nodes. Instead of direct expensive memory copying operations, the system uses virtual memory addresses that map to physical memory across the cluster, enabling efficient data sharing through the memory hierarchy and interconnect infrastructure without requiring costly explicit copy operations
Solution Approach 2:
The patent transitions from physical memory addressing to virtual memory addressing, adding an abstraction dimension. Virtual memory addresses serve as a unified namespace across all GPUs and nodes, allowing memory operations to be performed through address translation rather than physical data movement, thereby reducing copying overhead and improving scalability
2Productivity
If direct memory access between GPUs on different nodes is enabled, then memory sharing can occur, but the programming complexity increases
Solution Approach 1:
The patent implements a universal memory address space that works across all GPUs and nodes in the cluster. The same virtual memory address can refer to different physical memory locations on different nodes, providing a unified memory model that simplifies programming by eliminating the need for node-specific or GPU-specific memory access knowledge
Solution Approach 2:
The patent uses virtual memory address translation as a form of logical copying rather than physical data copying. The memory manager translates virtual addresses to physical addresses on the appropriate node, copying only the address mapping information rather than the actual data, thereby simplifying the programming interface while enabling memory sharing
3Productivity
If virtual memory sharing is implemented across nodes, then memory access efficiency improves, but the system complexity increases
Solution Approach 1:
The patent implements self-service memory management where the memory manager automatically handles virtual to physical address translation, memory allocation, and data placement. The system autonomously manages the complex mappings between virtual addresses and physical memory locations across nodes without requiring manual intervention or complex programming from the application developer
Data Source
AI summary
Apparatuses, systems, and techniques to facilitate memory management. In at least one embodiment, an application programming interface is performed to cause physical memory corresponding to shared virtual memory to be designated for use by a plurality of processors.