Load command processing method and system based on remote direct memory access protocol

Abstract

The present invention provides a load command processing method and system based on the Remote Direct Memory Access (RDMA) protocol. The system converts multiple graphics processor load commands into multiple source-side read commands at the source end, merges source-side read commands in the same queue pair, and transmits source-side write data to a target-side buffer. The target end then decomposes the source-side write data to generate multiple read commands and read back the data. The read-back data is merged into write data, and after processing, a target-side work queue item is generated. The work queue item is then read and the write data is transmitted to the source end. After receiving the target-side write data, the source end decomposes it into multiple read data and transmits them to the source-side graphics processor. The method and system provided by the present invention can transmit multiple merged read commands or data in a write manner over Ethernet while meeting the InfiniBand protocol, significantly improving bandwidth utilization, reducing the resource usage of the send queue, and maintaining the orderliness of system operations.

Description

Technical Field

[0001] The present invention relates to the technical field of computer hardware and network communication, and in particular to a load command processing method and system based on a remote direct memory access protocol. Background Art

[0002] With the rapid development of high-performance computing and artificial intelligence (AI) technologies, converged Ethernet protocols based on Remote Direct Memory Access (RDMA) have become a key technology for achieving low-latency, high-bandwidth communications in data centers and high-performance computing clusters. Specifically, Remote Direct Memory Access v2 (RDMA over converged Ethernet v2, RoCE v2) significantly improves data transmission efficiency in Ethernet environments by eliminating the overhead of traditional network protocol stacks, reducing latency and increasing throughput. However, existing RoCE v2 solutions still have some technical bottlenecks, particularly when Graphics Processing Units (GPUs) perform cross-chip loading operations via RoCE v2, resulting in low bandwidth utilization and wasted send queues (SQs).

[0003] Specifically, in the traditional RoCE v2 technology solution, a send queue stores work queue entries (WQEs). The RoCE Network Interface Card (RNIC) reads each WQE in turn and performs data transfers based on the transfer attributes and data length defined within it. If the GPU is to load directly via RNIC, each 64B / 128B GPU load command is converted into an Advanced eXtensible Interface (AXI) bus read command. This generates a read-only work queue entry and stores it in the corresponding send queue. The RNIC then reads the work queue entry and performs the read operation.

[0004] However, due to the limited send queue resources for storing work queue items, the multiple queue pairs (QPs) in RDMA quickly exhaust the number of send queue entries. Furthermore, when transmitting over Ethernet links based on remote direct memory access, the protocol header consumes bandwidth. In this case, increasing the amount of data carried increases bandwidth utilization. This leads to the desire to combine multiple GPU load commands into a single remote direct memory access network interface card read work queue entry to conserve send queue resources and increase bandwidth utilization. However, multiple read commands carry multiple read addresses, and read transfers in the InfiniBand (IB) protocol do not transmit data (payload) packets, making it impossible to transmit multiple addresses to the target chip.

[0005] Therefore, there is an urgent need for a technical solution that can effectively solve the problem of low bandwidth utilization and large occupation of send queue resources when the graphics processor performs cross-chip loading operations through RoCE v2. Summary of the Invention

[0006] In view of this, the present invention provides a load command processing method and system based on remote direct memory access protocol, so as to solve the above technical problems in the prior art.

[0007] According to one aspect of the present invention, a method for processing a load command based on a remote direct memory access protocol is provided, wherein the processing method includes:

[0008] Initiating multiple load commands through a source-side graphics processor, and converting the multiple load commands into multiple source-side read commands through a source-side advanced extensible interface bus, wherein the multiple source-side read commands include multiple queue pairs;

[0009] Merge source-side read commands in the same queue pair to generate source-side work queue items;

[0010] Transmitting the source-side write data included in the source-side work queue item to the target side via the Ethernet link;

[0011] Storing the source-side write data in the target-side receive buffer, where each entry of the source-side write data is stored in an entry in the target-side receive buffer;

[0012] Decomposing the write data in each entry of the target end receiving buffer to obtain multiple read commands;

[0013] Sending multiple read commands to the target-side advanced extensible interface bus, sending the multiple read commands to the target-side memory through the target-side advanced extensible interface bus and reading back the data;

[0014] Sending the read data to the target-side sending buffer via the target-side advanced extensible bus;

[0015] Merge the read-back data to generate a target-side work queue item;

[0016] Read the target end work queue item and transmit the target end write data included in the target end work queue item to the source end through the Ethernet link;

[0017] Receive target end write data;

[0018] Decomposing the target-side write data to obtain multiple read data, and transmitting the multiple read data to the source-side graphics processor.

[0019] According to another aspect of the present invention, a load command processing system based on a remote direct memory access protocol is provided, wherein the system comprises:

[0020] A source-side chip, comprising a source-side graphics processor, a source-side advanced extensible interface bus, a source-side remote direct memory access network interface card, a source-side merging generation module, and a source-side decomposition module;

[0021] A target-side chip, comprising a target-side remote direct memory access network interface card, a target-side decomposition module, a target-side memory, a target-side merge generation module, and a target-side advanced extensible interface bus;

[0022] The source-side graphics processor initiates multiple load commands and converts the multiple load commands into multiple source-side read commands via the source-side advanced extensible interface bus. The multiple source-side read commands include multiple queue pairs.

[0023] The source-side merge generation module merges source-side read commands in the same queue pair to generate source-side work queue items;

[0024] The source-side remote direct memory access network interface card reads the source-side work queue item and transmits the source-side write data included in the source-side work queue item to the target-side chip via the Ethernet link;

[0025] The target-side remote direct memory access network interface card receives the source-side write data through the Ethernet link, and stores the source-side write data in the target-side receive buffer, wherein each piece of the source-side write data is stored in an entry in the target-side receive buffer;

[0026] The target-side decomposition module decomposes the write data in each entry of the target-side receiving buffer into multiple read commands, and sends the multiple read commands to the target-side advanced extensible interface bus;

[0027] The target-side advanced extensible interface bus sends multiple read commands to the target-side memory and reads back the data, and then sends the read-back data to the target-side send buffer;

[0028] The target-side merge generation module merges the read-back data to generate a target-side work queue item;

[0029] The target-side remote direct memory access network interface card reads the target-side work queue item and transmits the target-side write data included in the target-side work queue item to the source-side chip via the Ethernet link;

[0030] The source-side remote direct memory access network interface card receives the target-side write data;

[0031] The source-side decomposition module decomposes the target-side write data into multiple read data and transmits the multiple read data to the source-side graphics processor.

[0032] According to another aspect of the present invention, an electronic device is provided, comprising: one or more processors and a memory, wherein the memory is used to store data; and the one or more processors are configured to implement the above method by reading and processing the data in the memory.

[0033] According to yet another aspect of the present invention, a computer network communication device is provided, which includes a storage medium and a data processing medium. When the data in the storage medium is executed by the processor, the processor executes the above method.

[0034] It can be seen from the above technical solutions that the technical solution provided by the present invention has at least the following advantages:

[0035] This invention combines multiple (≤8) GPU load commands, converted into multiple read commands on the Advanced eXtensible Interface (AEI) bus, into a single RDMA work queue entry with write semantics. Multiple (≤8) read addresses and related read information are used as write data. This write data is then sent to an Ethernet link via a Remote Direct Memory Access (RDMA) network interface card (NIC). At the target end, the write data is then decomposed into multiple read commands on the AEI bus. This improves RDMA bandwidth utilization and conserves work queue (memory dedicated to GPU loading / storage) resources when the GPU sends a large amount of load data using the Remote Direct Memory Access (RDMA) protocol. The technical solution provided by this invention is suitable for scenarios where a large number of GPU load commands are transmitted. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention, but do not constitute a limitation to the technical solution of the present invention.

[0037] Figure 1 A flow chart showing a processing method provided in an exemplary embodiment of the present invention;

[0038] Figure 2shows a structural diagram of a source-side read command in an exemplary embodiment of the present invention;

[0039] Figure 3 A schematic diagram showing the system composition in an exemplary embodiment of the present invention is shown;

[0040] Figure 4 A schematic diagram of a target-side remote direct memory access network interface card in an exemplary embodiment of the present invention is shown;

[0041] Figure 5 A schematic diagram of a remote direct memory access network interface card at the source end in an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] Various exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative and is not intended to limit the invention, its application, or use. The present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make this disclosure thorough and complete and to fully convey the scope of the invention to those skilled in the art.

[0043] Unless otherwise indicated, if the number of an element is not specifically limited, the element may be one or more. The term "plurality" means two or more, the term "based on" should be interpreted as "based at least in part on", and the terms "and / or" and "at least one of..." cover any one and all possible combinations of the listed items.

[0044] Please refer to Figure 1 , which shows a flowchart of a processing method provided by an exemplary embodiment of the present invention.

[0045] Specifically, the present invention provides a load command processing method based on the remote direct memory access protocol, wherein the processing method includes:

[0046] Initiating multiple load commands through a source-side graphics processor, and converting the multiple load commands into multiple source-side read commands through a source-side advanced extensible interface bus, wherein the multiple source-side read commands include multiple queue pairs;

[0047] Merge source-side read commands in the same queue pair to generate source-side work queue items;

[0048] Transmitting the source-side write data included in the source-side work queue item to the target side via the Ethernet link;

[0049] Storing the source-side write data in the target-side receive buffer, where each entry of the source-side write data is stored in an entry in the target-side receive buffer;

[0050] Decomposing the write data in each entry of the target end receiving buffer to obtain multiple read commands;

[0051] Sending multiple read commands to the target-side advanced extensible interface bus, sending the multiple read commands to the target-side memory through the target-side advanced extensible interface bus and reading back the data;

[0052] Sending the read data to the target-side sending buffer via the target-side advanced extensible bus;

[0053] Merge the read-back data to generate a target-side work queue item;

[0054] Read the target end work queue item and transmit the target end write data included in the target end work queue item to the source end through the Ethernet link;

[0055] Receive target end write data;

[0056] Decomposing the target-side write data to obtain multiple read data, and transmitting the multiple read data to the source-side graphics processor.

[0057] In an exemplary embodiment, the semantics of the source-side work queue item and the target-side work queue item are write operations.

[0058] In one exemplary embodiment, a source-side read command includes an address, a read identifier (RID), and a beat. These source-side read commands are stored in multiple entries in the source's send buffer. A beat of 0 indicates 64 bytes of data are read, while a beat of 1 indicates 128 bytes of data are read. The 64 bytes of data are stored in the lower address bits. Because read operations carry the read identifier during transmission over the Ethernet link, the memory and the Remote Direct Memory Access (RDMA) controller jointly ensure the order of read operations throughout the transmission process, preventing reordering. Specifically, the Remote Direct Memory Access (RDMA) network interface card (NIC) ensures that work queue entries are sent and received in the correct order. Read commands issued by the source parse the read address to determine the sequence number of each queue pair and then send them to the corresponding RDMA queue pair. Each source-side read command also carries the RID as a read identifier. These read commands are then combined and sent to the target as a write operation. The target parses the write data and decomposes it into a read command. For example, for a read command on the target side, arid=RID+QP means the read address identifier is set to the read identifier plus the queue pair sequence number. On the memory side, a reorder buffer is set up to maintain read order. The read data is broken down into read identifiers and queue pair sequence numbers, then transmitted to the corresponding queue pair on the target side. After that, it is combined into write data and transmitted back to the source side. Each read command is identified along the entire path using the read address identifier, read data identifier, and queue pair sequence number. Order is maintained by correctly identifying these identifiers and sequence numbers through the remote direct memory access network interface card and the memory.

[0059] In an exemplary embodiment, when transmitting via an Ethernet link, a pad is added at the end of each source read command to align each source read command 8B. The exemplary structure is as follows: Figure 2 shown.

[0060] In an exemplary embodiment, within a fixed time window, source-side read commands located in the same queue pair are retrieved from multiple entries in the source-side send buffer by indexing for merging operations, and the number n of source-side read commands merged each time is less than or equal to 8.

[0061] Please refer to Figures 3 to 5 , which respectively show schematic diagrams of a system, a target-side remote direct memory access network interface card, and a source-side remote direct memory access network interface card provided by exemplary embodiments of the present invention.

[0062] According to another aspect of the present invention, a load command processing system based on a remote direct memory access protocol is provided, comprising a source chip and a target chip. The source chip comprises a source graphics processor, a source advanced extensible interface bus, a source remote direct memory access network interface card, a source merge generation module, and a source decomposition module; and the target chip comprises a target remote direct memory access network interface card, a target decomposition module, a target memory, a target merge generation module, and a target advanced extensible interface bus.

[0063] In an exemplary embodiment, the overall processing flow of the load command by the system provided by the present invention is as follows:

[0064] 1. The source-side graphics processor initiates multiple load commands and converts them into multiple source-side read commands via the source-side Advanced eXtensible Interface bus. The multiple source-side read commands include multiple queue pairs.

[0065] 2. The source-side merge generation module merges source-side read commands located in the same queue pair to generate source-side work queue entries. In an exemplary embodiment, within a fixed time window, source-side read commands located in the same queue pair are retrieved from multiple entries in the source-side send buffer by indexing for merging. Furthermore, the number of source-side read commands merged each time is less than or equal to 8.

[0066] 3. The source-side remote direct memory access network interface card reads the source-side work queue entry and transmits the source-side write data contained in the source-side work queue entry to the target-side chip via the Ethernet link. In one exemplary embodiment, during transmission via the Ethernet link, a pad is added to the end of each source-side read command in the source-side work queue entry to ensure 8B alignment of each source-side read command.

[0067] 4. The target-side remote direct memory access network interface card receives the source-side write data through the Ethernet link and stores the source-side write data in the target-side receive buffer. Each piece of the source-side write data is stored in an entry in the target-side receive buffer.

[0068] 5. The target-side decomposition module decomposes the write data in each entry of the target-side receive buffer into multiple read commands and sends the multiple read commands to the target-side advanced extensible interface bus.

[0069] 6. The target-side advanced extensible interface bus sends multiple read commands to the target-side memory and reads back the data, and then sends the read-back data to the target-side send buffer.

[0070] 7. The target-side merge generation module merges the read-back data to generate a target-side work queue item.

[0071] 8. The target-side remote direct memory access network interface card reads the target-side work queue item and transmits the target-side write data included in the target-side work queue item to the source-side chip through the Ethernet link.

[0072] 9. The source-side remote direct memory access network interface card receives the target-side write data.

[0073] 10. The source-side decomposition module decomposes the target-side write data into multiple read data and transmits the multiple read data to the source-side graphics processor.

[0074] In an exemplary embodiment, the source-side write data is stored in the target-side receive buffer.

[0075] In an exemplary embodiment, the source-side write data is stored in a target-side receive first-in-first-out (RX FIFO) buffer.

[0076] In one exemplary embodiment, the write payload size for a source work queue entry with write semantics is 8B x n (n is the number of merged source read commands). Furthermore, when transmitting over an Ethernet link, the remote direct memory access network interface card reads the source work queue entry and, in accordance with the InfiniBand protocol, packages the write data and adds a header. Information that needs to be sent to the destination, such as the merge count, can be sent to the destination as part of the header using unused fields in the base transport header (BTH), such as the partition key (p_key).

[0077] It should be understood that Figures 3 to 5 The system, target-side remote direct memory access network interface card, and source-side remote direct memory access network interface card structure shown in the figure can correspond to the processing method described earlier in this specification. Therefore, the operations, features, and advantages described above for the processing method are also applicable to the system provided by the present invention and the components and modules included therein. The operations, features, and advantages described above for the system, target-side remote direct memory access network interface card, and source-side remote direct memory access network interface card structure and the components and modules included therein are also applicable to the processing method provided by the present invention. For the sake of brevity, certain operations, features, and advantages will not be repeated.

[0078] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of each module in the technical solution of the present invention may also be implemented as multiple modules, and / or at least some functions of multiple modules may be combined into a single module for implementation. The manner in which a specific module in the technical solution of the present invention performs an action includes the specific module itself performing the action, or being called or otherwise accessed by the specific module to perform the action (or performing the action in conjunction with the specific module). Therefore, the specific module that performs the action may include the specific module itself that performs the action and / or another module that is called or otherwise accessed by the specific module to perform the action.

[0079] In addition to the above technical solution, the present invention further provides an electronic device comprising one or more processors and a memory for storing data, wherein the one or more processors are configured to implement the above method by reading and processing data in the memory.

[0080] The present invention also provides a computer network communication device, which includes a storage medium and a data processing medium. When the data in the storage medium is executed by a processor, the processor executes the above method.

[0081] The technical solutions described in the present invention are not limited to the specific examples of the technical solutions described. The illustrations and descriptions of the present invention in the foregoing text and the accompanying drawings are not restrictive. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, the scope of protection claimed by the present invention is defined by the claims rather than the above description, and all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the scope of protection of the present invention.

Claims

1. A method for processing a load command based on a remote direct memory access protocol, characterized in that: The processing method comprises: Initiating multiple load commands through a source-side graphics processor, and converting the multiple load commands into multiple source-side read commands through a source-side advanced extensible interface bus, wherein the multiple source-side read commands include multiple queue pairs; Merge source-side read commands in the same queue pair to generate source-side work queue items; Transmitting the source-end write data included in the source-end work queue item to the target end via an Ethernet link; Storing the source-end write data in a target-end receive buffer, wherein each entry of the source-end write data is stored in an entry in the target-end receive buffer; Decomposing the write data in each entry of the target end receiving buffer to obtain multiple read commands; Sending the multiple read commands to a target-side advanced extensible interface bus, and sending the multiple read commands to a target-side memory through the target-side advanced extensible interface bus to read back data; Sending the read data to the target-side sending buffer via the target-side advanced extensible interface bus; Merge the read-back data to generate a target-side work queue item; Reading the target-side work queue item, and transmitting the target-side write data included in the target-side work queue item to the source end through the Ethernet link; receiving the target end write data; The target-end write data is decomposed to obtain a plurality of read data, and the plurality of read data are transmitted to the source-end graphics processor.

2. The processing method according to claim 1, characterized in that The semantics of the source-side work queue item and the target-side work queue item are write operations.

3. The processing method according to claim 1, characterized in that The source-end read command includes an address, a read identifier, and a beat, and the source-end read command is stored in an entry of a source-end sending buffer.

4. The processing method according to claim 1, characterized in that When transmitting via the Ethernet link, a filler is added at the end of each source-end read command to align each source-end read command 8B.

5. The processing method according to claim 3, characterized in that: Source-end read commands located in the same queue pair are retrieved from entries in the source-end send buffer by index within a fixed time window for merging, and the number of source-end read commands merged each time is less than or equal to 8.

6. A load command processing system based on remote direct memory access protocol, characterized in that: The system comprises: A source-side chip, comprising a source-side graphics processor, a source-side advanced extensible interface bus, a source-side remote direct memory access network interface card, a source-side merging generation module, and a source-side decomposition module; A target-side chip, comprising a target-side remote direct memory access network interface card, a target-side decomposition module, a target-side memory, a target-side merge generation module, and a target-side advanced extensible interface bus; The source-side graphics processor initiates multiple load commands and converts the multiple load commands into multiple source-side read commands via the source-side advanced extensible interface bus, wherein the multiple source-side read commands include multiple queue pairs; The source-side merging and generating module merges source-side read commands located in the same queue pair to generate a source-side work queue item; The source-side remote direct memory access network interface card reads the source-side work queue item, and transmits the source-side write data included in the source-side work queue item to the target-side chip via an Ethernet link; The target-side remote direct memory access network interface card receives the source-side write data through the Ethernet link, and stores the source-side write data in a target-side receive buffer, wherein each piece of the source-side write data is stored in an entry in the target-side receive buffer; The target-side decomposition module decomposes the write data in each entry of the target-side receiving buffer into multiple read commands, and sends the multiple read commands to the target-side advanced extensible interface bus; The target-side advanced extensible interface bus sends the multiple read commands to the target-side memory and reads back the data, and then sends the read-back data to the target-side sending buffer; The target-side merge generation module merges the read-back data to generate a target-side work queue item. The target-side remote direct memory access network interface card reads the target-side work queue item, and transmits the target-side write data included in the target-side work queue item to the source-side chip via the Ethernet link; The source-end remote direct memory access network interface card receives the target-end write data; The source-end decomposition module decomposes the target-end write data into multiple read data and transmits the multiple read data to the source-end graphics processor.

7. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing data; The one or more processors are configured to implement the method according to any one of claims 1 to 5 by reading and processing the data in the memory.

8. A computer network communication device, characterized in that: The device comprises a storage medium and a data processing medium. When the data in the storage medium is executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 5.

Images