Heterogeneous gpu system and data transfer method
A unified GPU memory operation interface and communication interface enable direct memory transfer across heterogeneous GPUs, addressing interoperability issues and improving performance in heterogeneous computing environments.
Patent Information
- Application Number
- GB2025020164
- Authority / Receiving Office
- GB · GB
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-02-25
- Filing Date
- 2025-11-27
- Publication Date
- 2026-11-18
AI Technical Summary
Existing communication libraries do not support direct memory transfer across GPUs from different manufacturers, limiting interoperability in heterogeneous computing environments.
A unified GPU memory operation interface and communication interface are configured to enable interoperable memory operations across heterogeneous GPUs, utilizing an access interface module that encapsulates native memory management interfaces and a data transfer module that enables direct data transfer through GDR and RDMA transfer primitives with dual-data buffers.
The solution achieves direct memory transfer across GPUs from different manufacturers, enhancing bandwidth and reducing latency, particularly in heterogeneous computing environments.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0002] Existing GPUs vary significantly in memory capacity, computing power, and architectural design. For example, NVIDIA Al00 GPU and NVIDIA Hl00 GPU respectively provide a 40 GB GPU memory capacity and an 80 GB GPU memory capacity, and AMD MI250X GPU provides a 128 GB GPU memory capacity. In practical application, especially in a heterogeneous computing cluster, GPUs from different manufacturers need to work together to complete data sharing and model training tasks. However, existing communication libraries (such as NVIDIA Collective Communication Library (NCCL), Radeon Open Compute (ROCm) Collective Communication Library (RCCL), and Unified Collective Communication Library (UCC)) only support data transfer across homogeneous GPUs, and cannot enable direct memory communication across GPUs from different manufacturers.
[0003] The existing communication libraries are primarily designed for homogeneous GPU environments, and lack support for direct memory transfer across heterogeneous GPUs. For example, NCCL is designed specifically for NVIDIA GPUs, relies on the Compute Unified Device Architecture (CUDA) ecosystem, and cannot support memory communication with GPUs from other manufacturers such as AMD and Intel. RCCL is designed based on the AMD ROCm ecosystem, supports collective communication across AMD GPUs, but cannot achieve direct memory transfer to NVIDIA or Intel GPUs. Unified Communication X (UCX) supports cross-platform communication, but it is still limited by manufacturer-specific drivers and interfaces in terms of direct GPU memory transfer. Message Passing Interface with GPUDirect RDMA (MPI-aware GDR) integrates the Remote Direct Memory Access (RDMA) technology to enable GPU memory communication, but it only supports GPUs from a same manufacturer and cannot achieve direct memory transfer across GPUs from different manufacturers. SUMMARY
[0004] In view of the foregoing problem, in the present application, a unified GPU memory operation interface and communication interface is configured, to support interoperable memory operations across heterogeneous GPUs. Technical solutions of the present application are as follows:
[0005] A heterogeneous GPU system includes an access interface module and a data transfer module.
[0006] The access interface module encapsulates native memory management interfaces of different GPUs and provides a unified memory access interface; and is configured with a GPU memory transfer interface based on one-sided semantics of libibverbs.
[0007] The data transfer module enables direct data transfer to a remote heterogeneous GPU memory through GDR and RDMA transfer primitives; and is configured with a dual-data buffer, a sender and a receiver respectively maintain two data buffers residing in GPU memory, and the buffers are allocated via the unified GPU memory access interface.
[0008] Preferably, the data transfer module is configured with two buffers, during data transfer, one buffer is configured to copy the GPU memory, and the other buffer is configured to transfer data over a network, and during a next cycle, the roles of the two buffers are swapped.
[0009] Preferably, during execution of a program, the access interface module is configured with memory management base class Memory, which is configured for managing memory of different GPU architectures.
[0010] HostMemory is configured for managing host memory; CudaMemory is configured for managing NVIDIA GPU memory; RocmMemory is configured for managing AMD GPU memory; and NeuwareMemory is configured for managing Cambricon Machine Learning Unit (MLU) memory.
[0011] Preferably, to achieve direct access to the GPU memory by a Host Channel Adapter (HCA), peer agent Application Programming Interfaces (APIs) provided by Mellanox Network Interface Cards (NICs) and OpenFabrics Enterprise Distribution (OFED) drivers thereof are used to enable the HCA to access a peer memory buffer, so as to establish a Direct Memory Access (DMA) data path from the Mellanox NIC to the GPU memory.
[0012] Host-to-device transfer is achieved through cudaMemcpyHostToDevice of CUDA, hipMemcpyHostToDevice of ROCm, or a corresponding API of MLU; device-to-host transfer is achieved through cudaMemcpyDeviceToHost of CUDA, hipMemcpyDeviceToHost of ROCm, or a corresponding API of MLU; and for device-to-device transfer, direct cross-GPU memory transfer is achieved via a Peripheral Component Interconnect Express (PCIe) bus.
[0013] Preferably, to achieve message synchronization between the sender and the receiver, a state buffer is configured, which includes writability information and a size of to-be-sent data. The buffer controls a data transfer process. State buffers are maintained on both sides of host memory, and messages are synchronized through one-sided write operations. After completing writing data to the receiver, the sender updates a message in the state buffer of the sender and sends the message to the receiver. After successfully detecting the message, the receiver starts to read sent data, to ensure synchronized and efficient data processing.
[0014] A heterogeneous GPU data transfer method includes the following steps: SI: initialization and configuration of a device: installing a module that supports GDR on a target host, invoking, by a user, a constructor of memory management base class Memory, and inputting a device type and device number, to complete configuration of a target device; S2: creation of a GPU memory management instance: after configuration of the device is completed, dynamically creating, by Memory, a corresponding GPU memory management class instance by a createMemoryClass method based on the device type specified by the user; S3: unified invocation of a GPU memory operation: directly completing, by the user, a GPU memory operation via a unified interface provided by Memory; and S4: data transfer using a pipelined dual-buffer strategy.
[0015] Preferably, in step SI, during initialization, whether the GDR module and an RDMA environment are correctly configured is checked, and necessary resources are initialized; and in step S3, after being allocated, GPU memory is registered to an RDMA subsystem via an ibv reg mr interface, to enable RDMA to directly access memory regions.
[0016] Preferably, in step S4, data transfer using the pipelined dual-buffer strategy includes: S41: copying, by a sender, a data block to a data buffer, and sending a message to a receiver to notify the receiver of a total size of to-be-sent data; S42: posting, by the sender, two write events on the data buffer, the first event being configured for writing actual data to remote GPU memory, and the second event being configured for synchronizing a readability message to the receiver, and the two events being posted sequentially on a same Queue Pair (QP), to ensure sequential execution of RDMA; and copying, by the sender, data into a second data buffer; S43: immediately copying, by the receiver, data from the data buffer of the receiver to an application program once detecting a readability notification, sending a writability notification to the sender, and beginning to receive network data by the second data buffer; and S44: after a current iteration is completed, immediately receiving a next readability notification.
[0017] Preferably, regardless of a data copy latency and an RDMA write latency, a next RDMA write to a same memory region occurs after copying is completed.
[0018] Preferably, if GPU memory copy latency is higher than RDMA write latency when a size of a message is less than a threshold, the size of the data buffer is configured to be close to the threshold to minimize transfer latency.
[0019] Compared to the prior art, the present application achieves the following beneficial effects:
[0020] In the present application, a direct memory transfer capability across heterogeneous GPU memory is achieved, and direct memory transfer across GPUs from Cambricon, Hygon, NVIDIA, and other manufacturers is supported.
[0021] The high-performance transfer solution designed in the present application achieves a higher bandwidth and lower latency compared to an unoptimized solution. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] FIG. 1 is a diagram of experimental results.
[0023] FIG. 2 is a flowchart of transfer. DESCRIPTION OF THE EMBODIMENTS
[0024] The technical solutions of the present application are described in detail below with reference to specific embodiments and the drawings. It should be understood that the embodiments of the present application and specific features in the embodiments are used for detailed description of the technical solutions of the present application, but are not intended to limit the technical solutions of the present application. Specific technical features can be combined with each other.
[0025] A heterogeneous GPU system includes an access interface module and a data transfer module. The access interface module encapsulates native memory management interfaces of different GPUs and provides a unified memory access interface; and is configured with a GPU memory transfer interface based on one-sided semantics of libibverbs. The data transfer module enables direct data transfer to a remote heterogeneous GPU memory through GDR and RDMA transfer primitives; and is configured with a dual-data buffer. A sender and a receiver respectively maintain two data buffers residing in GPU memory, and the buffers are allocated via the unified GPU memory access interface.
[0026] The interface encapsulates the native memory management interface of each type of GPU, and provides the unified memory access interface, such as memory allocation, memory copying, and copy release. In this manner, an upper-layer program can access memory of different GPU by the unified method, without regard for underlying hardware differences. During execution of a program, Unified Heterogeneous Memory (UHM) automatically adapts to a specific GPU type based on a type of a host device, to ensure seamless compatibility in a heterogeneous environment. Through this design, the upper-level program can migrate between different GPUs without modification, thereby simplifying development and maintenance complexity.
[0027] The unified memory operation interface for heterogeneous GPUs is as follows:
[0028] Definition and implementation of a core data structure:
[0029] In the present application, a unified memory management interface for heterogeneous GPUs is configured, which masks memory management details of different GPU architectures and provides users with an efficient and user-friendly memory management solution. The core data structure includes a memory management base class MemoryBase. This base class defines a base memory management interface, which includes functions such as memory allocation and release, and data transfer. Specific subclasses, such as CudaMemory and RocmMemory, implement these interfaces based on different GPU architectures.
[0030] Unified interface class Memory provides a user-friendly interface, which is configured for managing memory of different GPU architectures. A user can automatically create a corresponding GPU memory management instance by setting a device type. For different GPU architectures, the present application implements the following memory management classes:
[0031] HostMemory is configured for managing host memory; CudaMemory is configured for managing NVIDIA GPU memory; RocmMemory is configured for managing AMD GPU memory; and NeuwareMemory is configured for managing Cambricon MLU memory.
[0032] In terms of interface implementation and function expansion, the memory allocation and release interface adopts corresponding APIs based on different GPU architectures. For NVIDIA GPUs, CUDA APIs (such as cudaMalloc and cudaFree) are used to implement GPU memory allocation and release. For AMD GPUs, ROCm APIs (such as hipMalloc and hipFree) are used to implement GPU allocation and release. For Cambricon MLUs, MLU APIs (such as mluMalloc and mluFree) are used to implement MLU memory allocation and release.
[0033] In terms of data cudaMemcpyHostToDevice corresponding API of cudaMemcpyDeviceToHost transfer interfaces, host-to-device transfer of CUDA, hipMemcpyHostToDevice MLU; device-to-host transfer is of CUDA, hipMemcpyDeviceToHost is achieved of ROCm, achieved of ROCm, through or a through or a corresponding API of MLU; and for device-to-device transfer, direct cross-GPU memory transfer is achieved via a PCIe bus, to avoid additional overheads of data passing through the host memory.
[0034] In terms of initialization and resource management, during initialization, each GPU memory management class (such as CudaMemory) sets a current device and initializes corresponding GPU context. The user can automatically complete creation and initialization of a GPU memory management instance by an init method of the Memory class.
[0035] (1) The user needs to install a module that supports GDR on a target host, such as a NVIDIA GDR module or an AMD peermem module. These modules typically integrated with GPU drivers and OFED drivers, and can provide necessary support for RDMA communication to operate a GPU memory.
[0036] (2) The user creates a communication endpoint instance through a GPU memory communication endpoint class. During instantiation, the communication endpoint class automatically detects a GDR environment, and prepares for subsequent GPU memory operations and RDMA communication.
[0037] (3) The user allocates a GPU memory via a memory operation interface provided by the communication endpoint class. After allocation is completed, the communication endpoint class invokes an ibv reg mr interface to register the GPU memory to an RDMA subsystem. This process ensures that RDMA can directly access the GPU memory, to achieve efficient data transfer between GPU memory.
[0038] (4) The user can implement direct read / write operations on a peer GPU memory via RDMA primitive interfaces (such as rdma write and rdma read) encapsulated by the communication endpoint class. These operations can significantly lower data transfer latency and system overheads based on one-sided semantics of RDMA.
[0039] A heterogeneous GPU memory operation process is as follows:
[0040] 1 : Initialization and configuration of a device:
[0041] The user invokes a constructor of the Memory class and enters a device type (such as MemoryType::NVIDIA_GPU or MemoryType::AMD_GPU) and device number, to complete configuration of a target device. This process ensures that the GPU memory management module can correctly identify a type of the target device and allocate a corresponding resource to the target device.
[0042] 2: Creation of a GPU memory management instance:
[0043] After configuration of the device is completed, the Memory class dynamically create a corresponding GPU memory management class instance by a createMemoryClass method based on the device type specified by the user. For example, for a NVIDIA GPU, a CudaMemory instance is created; and for an AMD GPU, a RocmMemory instance is created. This mechanism achieves dynamic adaptation for memory management across different GPU architectures.
[0044] 3 : Unified invocation of a GPU memory operation:
[0045] The user can directly complete a GPU memory operation via a unified interface (such as allocate buffer, free buffer, or copy host to device) provided by the Memory class, without regard for underlying implementation details. This design greatly reduces complexity for the user and ensures efficiency and consistency of GPU memory operations.
[0046] 4 : Direct point-to-point transfer solution for heterogeneous GPU memory
[0047] A GDR module is a technology for implementing direct memory transfer across GPUs. Through this module, a host channel adapter can directly access the GPU memory without going through a conventional CPU memory copy process. This not only lowers data transfer latency, but also enhances bandwidth utilization, making it particularly suitable for communication between GPUs in high-performance computing and distributed systems. GDR modules are typically provided by GPU manufacturers, such as the NVIDIA GDR module and the AMD peermem module, which are integrated with OFED drivers, to provide underlying support for RDMA communication.
[0048] Based on the GDR module and the libibverbs library for RDMA, in the present application, a GPU memory communication endpoint class is configured, which encapsulates GPU memory registration, Queue Pair (QR) creation, and primitive operations for data transfer. This class primarily includes the following core functions:
[0049] Initialization and resource management: During initialization, this class checks whether the GDR module and an RDMA environment are correctly configured, and initializes necessary resources such as a Protection Domain (PD) and a QP.
[0050] GPU memory operation interface: The user can easily allocate and release a GPU memory via this interface. After being allocated, the GPU memory is registered to an RDMA subsystem via an ibv reg mr interface, to enable RDMA to directly access these memory regions.
[0051] RDMA data transfer: This class encapsulates one-sided operations (such as rdma write and rdma read) of RDMA, to allow the user to directly read from / write to a peer GPU memory. These operations ensure efficiency and reliability of data transfer based on native semantics of RDMA.
[0052] Through the GPU memory communication endpoint class, the user can seamlessly integrate GPU memory operations with RDMA communication, thereby simplifying implementation of direct memory transfer across GPUs.
[0053] Pipelined dual-buffer data transfer strategy:
[0054] In a heterogeneous GPU environment, a significant performance difference exists between direct GPU memory read / write and GPU memory copying. GPU memory read / write typically is implemented through RDMA, and features low latency and high bandwidth. However, GPU memory copying may involve more system overheads, especially when data needs to be transferred between GPUs. In addition, registration of large blocks of the GPU memory for buffers may lead to waste of GPU memory resources, particularly when GPU memory resources are limited. Therefore, it is particularly important to design a transfer strategy that can efficiently utilize GPU memory resources and balance GPU memory copy latency and transfer latency. FIG. 2 shows a pipelined dual-buffer data transfer strategy designed in the present application.
[0055] Stage 1: A sender copies a data block to a data buffer, and sends a message to a receiver to notify the receiver of a total size of to-be-sent data. The sender and the receiver can calculate a number of required transfer iterations based on this information.
[0056] Stage 2: The sender posts two write events on the data buffer. The first event is configured for writing actual data to remote GPU memory, and the second event is configured for synchronizing a readability message to the receiver. The two events are posted sequentially on a same QP, to ensure sequential execution of RDMA. Meanwhile, the sender starts to copy data into the second data buffer. In this manner, RDMA data transfer and message synchronization are overlapped with GPU memory copying.
[0057] Stage 3: The receiver immediately copies data from the data buffer of the receiver to an application program once detecting a readability notification, and sends a writability notification to the sender. Meanwhile, network data are received into the second data buffer. Regardless of data copy latency and RDMA write latency, a next RDMA write to a same memory region occurs after copying is completed. This ensures that data is not overwritten, thereby ensuring integrity of the data.
[0058] After a current iteration is completed, a next readability notification is immediately received (for the second buffer), so that efficiency of the pipeline is maximized. If GPU memory copy latency is higher than RDMA write latency when a size of a message is less than a threshold, the size of the data buffer is configured to be close to the threshold to achieve the foregoing objective. Through this pipelined design, RDMA data transfer latency is hidden in GPU memory copy latency, thereby significantly lowering network transfer latency for a large data block.
[0059] In this solution, two key components, namely, a state buffer (configured for message synchronization) and a dual-data buffer (configured for pipelined data transfer). These two components respectively reside in host memory and GPU memory.
[0060] The state buffer is configured for message synchronization. To achieve message synchronization between the sender and the receiver, a state buffer is configured in the present application, which includes writability information and a size of to-be-sent data. The buffer controls a data transfer process. Because the GPU memory cannot be directly accessed by a Central Processing Unit (CPU) program, during communication, it is not possible to directly embed a transfer control message into data, unlike the polling design of FaRM. In contrast, in this solution, state buffers are remained on both sides of the host memory, and messages are synchronized through one-sided write operations. After completing writing data to the receiver, the sender updates a message in the state buffer of the sender and sends the message to the receiver. After successfully detecting the message, the receiver starts to read sent data, to ensure synchronized and efficient data processing.
[0061] The dual-data buffer is configured for pipelined data transfer. The sender and the receiver respectively maintain two data buffers residing in the GPU memory, which are configured to store actual data. These buffers are allocated via a unified GPU memory access interface. During data transfer, one buffer is configured for copying the GPU memory, and the other buffer is configured for transferring data over a network. During a next cycle, the roles of the two buffers are swapped. This dual-buffer design enables overlapping of RDMA data transfer and GPU memory copying, thereby achieving pipelined transfer.
[0062] To achieve direct access to the GPU memory by an HCA, peer agent APIs provided by Mellanox NICs and OFED drivers thereof are used. Through the APIs, the HCA can access a peer memory buffer, to establish a DMA data path from the Mellanox NIC to the GPU memory. The GPU driver supports this function by implementing a peer memory client (static struct peer memory client ex). These modules are commonly referred to as GDR. GPU manufacturers (such as NVIDIA and AMD) provide corresponding kernel modules to support these functions. For example, a NVIDIA driver includes a GDR module, and a peermem module is embedded into an AMD driver. However, existing direct GPU memory transfer solutions (such as xCCL, UCX, and MPI-aware GDR) support only data transfer across homogeneous GPUs. To overcome this limitation, GDR and the libibverbs library for RDMA are extended and integrated into a unified memory access interface. Heterogeneous GPU memory is registered via a libibverbs interface (such as ibv reg mr), and direct data transfer to the remote heterogeneous GPU memory through GDR and RDMA transfer primitives.
[0063] In the present application, an experiment was designed to validate that the unified heterogeneous GPU memory access method of the present application significantly enhances data transfer efficiency. In this experiment, four different data transfer solutions were compared, which were Ideal (an ideal solution): using RDMA writing only, with no additional copy latency; G2H2G (a host memory relay solution): using host memory as a relay to achieve transfer between GPU memory; NoPipe (a serial copy / transfer solution): non-pipelined version of the UHM method of the present application; and UHM (the method of the present application).
[0064] Experimental results are shown in FIG. 1. In a case of a small data volume (less than 8,192 bytes), transfer latency of UHM is equivalent to that of the NoPipe solution and is significantly lower than that of the G2H2G solution. Average latency of UHM and average latency of the NoPipe solution are lowered by approximately 50% to 70% compared to that of the G2H2G solution. In a case of a large data volume (greater than 32,768 bytes), latency of UHM is significantly lower than that of the G2H2G solution and the NoPipe solution. The experimental data demonstrates that average latency of UHM is lowered by 38.1% to 87.2% compared to that of the G2H2G solution, and is lowered by 23.5% to 76.7% compared to that of the NoPipe solution. The performance of UHM is approaching the theoretical limit of the Ideal solution. By adopting the pipelined dual-data buffer design, UHM significantly lowers latency caused by memory copying. Especially in a big data transfer scenario, a performance optimization effect of UHM is particularly significant, fully demonstrating superiority of UHM in enhancing data transfer efficiency. These results demonstrate that the unified heterogeneous GPU memory access method of the present application can effectively lower data transfer 5 latency and significantly enhance system performance, especially in a complex heterogeneous environment.
[0065] The above are merely preferred implementations of the present application. It should be noted that those of ordinary skill in the art may further make several improvements and modifications without departing from the technical principle of the present application, and 10 these improvements and modifications shall fall within the scope of protection of the present application.
Claims
WHAT IS CLAIMED IS:
1. A heterogeneous Graphics Processing Unit (GPU) system, comprising an access interface module and a data transfer module,the access interface module encapsulating native memory management interfaces of different GPUs and providing a unified memory access interface; and being configured with a GPU memory transfer interface based on one-sided semantics of libibverbs; andthe data transfer module enabling direct data transfer to remote heterogeneous GPU memory through GPUDirect RDMA (GDR) and Remote Direct Memory Access (RDMA) transfer primitives; and being configured with a dual-data buffer, a sender and a receiver respectively maintaining two data buffers residing in GPU memory, and the buffers being allocated via the unified GPU memory access interface.
2. The heterogeneous GPU system according to claim 1, wherein the data transfer module is configured with two buffers, during data transfer, one buffer is configured to copy the GPU memory, and the other buffer is configured to transfer data over a network, and during a next cycle, the roles of the two buffers are swapped.
3. The heterogeneous GPU system according to claim 1, wherein during execution of a program, the access interface module is configured with memory management base class Memory, which is configured for managing memory of different GPU architectures; andHostMemory is configured for managing host memory; CudaMemory is configured for managing NVIDIA GPU memory; RocmMemory is configured for managing AMD GPU memory; and NeuwareMemory is configured for managing Cambricon Machine Learning Unit (MLU) memory.
4. The heterogeneous GPU system according to claim 1, wherein to achieve direct access to the GPU memory by a Host Channel Adapter (HCA), peer agent Application Programming Interfaces (APIs) provided by Mellanox Network Interface Cards (NICs) and OpenFabrics Enterprise Distribution (OFED) drivers thereof are used to enable the HCA to access a peer memory buffer, so as to establish a Direct Memory Access (DMA) data path from the Mellanox NIC to the GPU memory; andhost-to-device transfer is achieved through cudaMemcpyHostToDevice of Compute Unified Device Architecture (CUD A), hipMemcpyHostToDevice of Radeon Open Compute (ROCm), or a corresponding API of MLU; device-to-host transfer is achieved through cudaMemcpyDeviceToHost of CUD A, hipMemcpyDeviceToHost of ROCm, or a corresponding API of MLU; and for device-to-device transfer, direct cross-GPU memory transfer is achieved via a Peripheral Component Interconnect Express (PCIe) bus.
5. The heterogeneous GPU system according to claim 1, wherein to achieve message synchronization between the sender and the receiver, a state buffer is configured, which comprises writability information and a size of to-be-sent data, and controls data transfer; state buffers are maintained on both sides of a host memory, and messages are synchronized through one-sided write operations; after completing writing data to the receiver, the sender updates a message in the state buffer of the sender and send the message to the receiver, and after successfully detecting the message, the receiver starts to read sent data, to ensure synchronized and efficient data processing.
6. A heterogeneous GPU data transfer method, comprising the following steps:SI: initialization and configuration of a device:installing a module that supports GDR on a target host, invoking, by a user, a constructor of a memory management base class Memory, and inputting a device type and device number, to complete configuration of a target device;S2: creation of a GPU memory management instance:after configuration of the device is completed, dynamically creating, by Memory, a corresponding GPU memory management class instance by a createMemoryClass method based on the device type specified by the user;S3: unified invocation of a GPU memory operation:directly completing, by the user, a GPU memory operation via a unified interface provided by Memory; andS4: data transfer using a pipelined dual-buffer strategy.
7. The heterogeneous GPU data transfer method according to claim 6, wherein in step SI, during initialization, whether the GDR module and an RDMA environment are correctly configured is checked, and necessary resources are initialized; andin step S3, after being allocated, GPU memory is registered to an RDMA subsystem via an ibv reg mr interface, to enable RDMA to directly access memory regions.
8. The heterogeneous GPU data transfer method according to claim 6, wherein in step S4, data transfer using the pipelined dual-buffer strategy comprises:S41: copying, by a sender, a data block to a data buffer, and sending a message to a receiver to notify the receiver of a total size of to-be-sent data;S42: posting, by the sender, two write events on the data buffer, the first event being configured for writing actual data to a remote GPU memory, and the second event being configured for synchronizing a readability message to the receiver, and the two events being posted sequentially on a same Queue Pair (QP), to ensure sequential execution of RDMA; and copying, by the sender, data to a second data buffer;S43: immediately copying, by the receiver, data from the data buffer of the receiver to an application program once detecting a readability notification, sending a writability notification to the sender, and beginning to receive network data by the second data buffer; andS44: after a current iteration is completed, immediately receiving a next readability 5 notification.
9. The heterogeneous GPU data transfer method according to claim 6, wherein a next RDMA write to a same memory region occurs after copying is completed.
10. The heterogeneous GPU data transfer method according to claim 6, wherein if GPU memory copy latency is higher than RDMA write latency when a size of a message is less than a 10 threshold, a size of a data buffer is configured to be close to the threshold to minimize transfer latency.A
Citation Information
Patent Citations
ViewCN119226193AonEspacenetopensinnewtab
