RDMA Inter-Node Communication for Shared GPU Containers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cluster management tools waste GPU computing power due to the inability of a single GPU to be occupied by multiple containers, leading to CPU overload from communication between different containers, especially in complex tasks.
Innovation Solution
A method utilizing RDMA modules for direct memory access between nodes, avoiding CPU involvement by determining node identities through memory information and performing communication actions directly between source and target nodes, including identity document allocation and unified memory address editing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtualization technologies are adopted to share GPU resources among multiple containers, then GPU resource utilization is improved, but CPU occupation increases due to communication between containers
Solution Approach 1:
The patent extracts the communication processing function from the CPU by introducing a dedicated RDMA module. The RDMA module handles direct memory access between containers without involving the CPU, thereby reducing CPU occupation time while maintaining GPU resource sharing capabilities.
Solution Approach 2:
The RDMA module serves as an intermediary between source and target containers for memory access operations. It mediates the communication process by enabling direct memory access through hardware-level support, eliminating the need for CPU intervention in the actual data transmission.
2Reliability
If a single GPU is assigned to one container, then container isolation is maintained, but GPU computing power is wasted
Solution Approach 1:
The patent segments the GPU into multiple virtual GPU instances using virtualization technologies. Each virtual GPU can be independently assigned to different containers, allowing multiple containers to share the physical GPU resource while maintaining logical isolation and independent access.
Solution Approach 2:
The virtualized GPU system enables a single physical GPU to serve multiple containers simultaneously, making the GPU resource universal. The GPU can be dynamically allocated to different containers based on workload requirements, improving overall utilization while maintaining container isolation through virtualization layers.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances communication efficiency by reducing CPU occupation during node interactions, allowing zero-copy data transmission and improving overall cluster management efficiency.
Implementation Method 1
a communication action is performed on the first source node and the first target node through a remote direct memory access (RDMA) module according to the identity document of the first source node and the identity document of the first target node
Data Source
AI summary
The present disclosure provides a method for communication between nodes. The method includes: obtaining a list of field-programmable gate arrays; virtualizing the field-programmable gate arrays according to the list of field-programmable gate arrays to obtain a first source node and a first target node; obtaining an application descriptor between the first source node and the first target node, the application descriptor includes a memory address of the first source node and a memory address of the first target node; obtaining an identity document of the first source node according to the memory address, and obtaining an identity document of the first target node according to the memory address; and performing a communication action on the first source node and the first target node through a remote direct memory access module according to the identity document of the first source node and the identity document of the first target node.