Shared Virtual Memory API Using Multicast Handles for GPU Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face inefficiencies in sharing memory between GPUs across different nodes in a GPU cluster, requiring expensive operations to copy memory and complicating programming efforts.
Innovation Solution
Implementing an application programming interface (API) to allocate, designate, and manage shared virtual memory across processors, utilizing multicast memory handles to facilitate efficient memory sharing and access among GPUs in a cluster.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If memory is copied between GPUs on different nodes, then memory contents can be transferred, but resource usage and time consumption increase significantly
Solution Approach 1:
The patent merges memory spaces across multiple GPUs by creating a unified shared virtual memory system. Instead of copying memory between separate GPU memory spaces, the system allows direct access to the same memory region from multiple GPUs, eliminating redundant data transfer operations and reducing resource consumption.
Solution Approach 2:
The patent introduces a memory management unit as an intermediary that handles memory allocation, address translation, and access coordination. This mediator enables efficient memory sharing by managing the complexity of cross-node memory access transparently, allowing GPUs to access shared memory without direct copying operations.
2Reliability
If memory is copied between GPUs on different nodes, then memory contents can be transferred, but the operation becomes expensive and time-consuming
Solution Approach 1:
The patent merges memory spaces across multiple GPUs by creating a unified shared virtual memory system. Instead of copying memory between separate GPU memory spaces, the system allows direct access to the same memory region from multiple GPUs, eliminating redundant data transfer operations and reducing resource consumption.
Solution Approach 2:
The patent performs preliminary memory setup by pre-allocating and pre-configuring shared memory regions before computation tasks begin. This advance preparation eliminates the need for time-consuming memory copying operations during execution, as memory is already in place and ready for direct access.
3Productivity
If direct memory sharing between GPUs is implemented, then resource efficiency improves, but programming complexity increases
Solution Approach 1:
The patent introduces a memory management unit as an intermediary that handles memory allocation, address translation, and access coordination. This mediator enables efficient memory sharing by managing the complexity of cross-node memory access transparently, allowing GPUs to access shared memory without direct copying operations.
Solution Approach 2:
The patent implements automatic memory management features where the system handles memory allocation, address translation, and access coordination without requiring explicit programmer intervention. The memory management unit automatically manages the shared memory space, reducing programming complexity while maintaining resource efficiency.
Data Source
AI summary
Apparatuses, systems, and techniques to facilitate memory management. In at least one embodiment, an application programming interface is performed to cause physical memory corresponding to shared virtual memory to be designated for use by a plurality of processors.