GPU RDMA Communication via Pre-generated NIC Command Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
GPUs in multi-node clusters cannot efficiently initiate RDMA communications without CPU intervention, leading to inefficient GPU-GPU data transfer and requiring CPU involvement, which contradicts the goal of freeing the CPU for other tasks.
Innovation Solution
Pre-generating a queue of communication command packets using a CPU, allowing GPUs to signal an NIC to execute RDMA commands by decrementing a counter variable, enabling the NIC to transmit data to a remote GPU without continuous CPU oversight.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a CPU-based communication library is used to communicate between distributed GPUs, then communication between GPUs can be achieved, but the CPU is occupied and cannot perform other functions
Solution Approach 1:
The patent extracts the communication command generation function from the CPU and transfers it to the GPU. The GPU independently generates RDMA communication commands without CPU intervention, while the CPU retains only the initial work queue allocation function. This resolves the contradiction by freeing the CPU for other tasks while maintaining GPU-to-GPU communication capability.
Solution Approach 2:
The GPU is empowered to self-generate RDMA communication commands autonomously without requiring CPU-based communication libraries. By implementing command generation logic directly in the GPU, the system achieves self-service communication, eliminating the need for CPU involvement in the communication critical path.
2Productivity
If GPUs generate communication network commands independently, then CPU can be freed for other tasks, but the operation becomes serial and cannot be effectively parallelized
Solution Approach 1:
The patent segments the command generation process into two independent parallel streams: one for read commands and one for write commands. Each stream can be processed independently by the GPU, enabling parallelization of command generation while maintaining CPU availability. This segmentation resolves the contradiction between independence and parallelization capability.
3Speed
If RDMA commands are executed without pre-generated queues, then real-time communication is possible, but continuous CPU oversight is required
Solution Approach 1:
The patent implements preliminary action by having the CPU pre-generate and queue RDMA communication commands before GPU execution. These pre-generated commands are stored in a command buffer that the GPU can access and execute autonomously. This approach maintains high-speed RDMA transfer while reducing the control mechanism to simple command retrieval and execution, eliminating the need for continuous CPU oversight.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, devices, and systems for transmitting data over a computer communications network are disclosed. A queue of communications commands can be pre-generated using a central processing unit (CPU) and stored in a device memory of a network interface controller (NIC). Thereafter, if a graphics processing unit (GPU) has data to communicate to a remote GPU, it can store the data in a send buffer, where the location in the buffer is pointed to by a pre-generated command. The GPU can then signal to the interface device that the data is ready, triggering execution of the pre-generated command to send the data.