RDMA Hardware Offload for Small-Block Data Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing RDMA data transmission methods in scenarios like MoE models consume excessive CPU/GPU resources due to the need for generating and managing numerous small data blocks, leading to performance bottlenecks.
Innovation Solution
Implementing a hardware offload asynchronous copy instruction set within an RDMA network device to offload data transmission tasks to dedicated hardware engines, reducing dependence on CPU/GPU resources by using multi-level synchronous control data transmission instructions and direct memory access technology.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If CPU/GPU resources are used to generate and manage numerous small data blocks for RDMA transmission, then data transmission can be performed, but system performance deteriorates due to excessive resource consumption
Solution Approach 1:
The patent extracts the data transmission task from CPU/GPU resources and transfers it to a dedicated hardware engine within the network device. The hardware engine independently handles work element assembly, data block generation, and RDMA transmission, while CPU/GPU resources focus on computation tasks, thereby resolving the resource consumption problem
Solution Approach 2:
The patent introduces a hardware engine as an intermediary component between CPU/GPU and the RDMA transmission process. This hardware engine receives computation results from CPU/GPU, autonomously manages data block generation and transmission queue management, and completes RDMA transmission without requiring continuous CPU/GPU intervention
2Productivity
If CPU/GPU resources are used for RDMA data transmission management, then data blocks can be transmitted, but data delivery delays increase
Solution Approach 1:
The hardware engine maintains continuous operation for data transmission tasks without being interrupted by CPU/GPU scheduling cycles. It continuously generates data blocks, manages transmission queues, and performs RDMA operations, eliminating the time losses associated with context switching and resource allocation that occur when using general-purpose processors
Solution Approach 2:
The hardware engine is self-sufficient in handling data transmission operations. It autonomously assembles work elements, generates data blocks, manages transmission queues, and executes RDMA operations without requiring CPU/GPU intervention, thereby reducing processing delays and improving data delivery speed
3Productivity
If CPU/GPU resources are relied upon for RDMA transmission, then data blocks can be managed, but system performance bottlenecks occur in data-intensive scenarios
Solution Approach 1:
The patent segments the system into distinct functional components: CPU/GPU for computation, hardware engine for data transmission management, and RDMA network interface for network operations. This segmentation allows each component to specialize in its designated task, improving overall system performance in data-intensive scenarios while maintaining manageable complexity through clear functional boundaries
Data Source
AI summary
An RDMA data transmission method, a network device and a network system is provided. An implementation of the method is applied to a network device comprising an xPU and an RNIC, the xPU comprises a first engine, and the RNIC comprises a second engine in communication with the first engine, a WQE Buffer, and an RDMA engine in communication with the second engine. The method comprises: assembling, by the first engine, a WE based on a hardware offload asynchronous copy instruction set, and transmitting the WE to the second engine; storing, by the second engine, the WE into the WQE Buffer and transmitting the WE to the RDMA engine; performing, by the RDMA engine, data processing based on the WE, the data processing comprising data transmitting, memory accessing and queue managing; and receiving, by the first engine, a feedback of the data processing transmitted via the second engine.