A data communication method, system and related devices

Abstract

The application provides a data communication method, system and related equipment. The method comprises the following steps: in an information exchange stage, a first node pre-registers a first memory domain to RDMA hardware, so that the RDMA hardware can directly access the first memory domain through an RDMA technology, then divides the first memory domain into a plurality of memory blocks, and records address information of the memory blocks; before communication, the first node and a second node exchange address information of blocks of respective memory domains; in a communication stage, each communication can directly perform RDMA read-write operation based on address information of memory blocks of the second node, each communication does not need to perform transmission preparation operation, that is, does not need to copy data, does not need to register a memory domain, and does not need to send a handshake message, so that the communication efficiency can be greatly improved in a frequent point-to-point communication scenario.

Description

Technical Field

[0001] This application relates to the field of communications, and more particularly to a data communication method, system and related equipment. Background Technology

[0002] Remote direct memory access (RDMA) is a technology that allows direct access from the memory of one computing device to the memory of another computing device without going through the operating system or CPU of the computing device. This improves the efficiency of data transfer, reduces network latency and CPU load, and is widely used in various scenarios.

[0003] However, RDMA technology requires transmission preparation operations before each data transmission, such as registering memory regions, copying data in buffers, or exchanging handshake messages. This is because the destination address for each data write is different. Even in point-to-point communication scenarios where the sender and receiver remain the same, the memory address for each data write may be different. Therefore, RDMA requires transmission preparation operations every time. When communication between the sender and receiver is frequent, frequent transmission preparation operations will affect communication efficiency and increase communication latency. Summary of the Invention

[0004] This application provides a data communication method, system, and related equipment to solve the problem that in point-to-point communication scenarios, the sending end and the receiving end need to perform transmission preparation operations for each RDMA, which affects communication efficiency and increases communication latency.

[0005] In a first aspect, a data communication method is provided, which includes the following steps: in the information exchange phase, a first node obtains exchange information from a second node, the exchange information including the address information of a memory block, the memory block being obtained by partitioning the memory domain of the second node, the memory domain of the second node being used for remote direct memory access (RDMA) hardware; in the communication phase, the first node communicates with the second node multiple times through the first node's RDMA hardware, each communication using the exchange information to read or write data from the memory block.

[0006] Implementing the method described in the first aspect, during the information exchange phase, the first node pre-registers a first memory domain to the RDMA hardware, enabling the RDMA hardware to directly access the first memory domain via RDMA technology. Then, the first memory domain is divided into multiple memory blocks, and the address information of each block is recorded. Thus, when the first node and the second node have communication needs, they can exchange the address information of their respective memory domain blocks before communication. During the communication phase, when the first node and the second node begin RDMA communication, each communication can directly perform RDMA read and write operations based on the address information of the blocks in the second node's memory domain. Each communication does not require transmission preparation operations, i.e., no data copying, no memory domain registration, and no handshake message sending. This significantly improves communication efficiency in frequent point-to-point communication scenarios.

[0007] In one possible implementation, the method further includes the following steps: a first node registers a first memory domain of a first capacity to the first node's RDMA hardware, the first memory domain being used for RDMA communication by the first node's RDMA hardware, the first capacity being not less than the data size of a single RDMA communication by the first node, the first node dividing the first memory domain into multiple first memory blocks and obtaining address information for each first memory block, the first node generating exchange information based on the address information of each first memory block, the first node sending the exchange information to a third node, the exchange information being used by the third node for multiple communications with the first node through the third node's RDMA hardware.

[0008] Understandably, in traditional technologies, the size of the registered memory domain is determined by the size of a single RDMA data transfer. Each registered memory domain is used for the current communication, requiring memory domain registration for every communication, wasting system resources and impacting communication efficiency. The implementation described in this application registers a first memory domain during the information exchange phase before the communication phase, sufficient for multiple point-to-point communications. The first memory domain's capacity is not less than the data size of a single RDMA communication, allowing multiple RDMAs to utilize the pre-registered memory domain. This avoids the need for MR registration for each RDMA, reducing system resource waste.

[0009] In one possible implementation, the first node includes a first application and a first communication library, and the second node includes a second application. There is a communication requirement between the first application and the second application. The first node generates exchange information based on the address information of each memory block, including the following steps: the first application of the first node sends a request to the first communication library to register a memory domain; the first communication library of the first node allocates at least one first memory block from multiple first memory blocks for the first application to use based on the request to register the memory domain; the first communication library of the first node generates exchange information based on the address information of at least one first memory block; the exchange information of the first node includes at least the address information of at least one first memory block of the first application.

[0010] In practical implementation, the communication library is a software library used to manage and optimize data exchange between different computing nodes or processes. It provides developers with data transmission capabilities across processes, nodes, or networks. Communication libraries are typically used in distributed systems, high-performance computing, parallel computing, data centers, and cloud computing scenarios. The communication library can include multiple API functions to implement different operational logics, such as network communication, memory management, and data transmission, enabling developers to efficiently and easily implement inter-process communication or cross-node data transmission. Assuming the traditional API function for registering memory domains is the first function, and the API function for registering memory domains in this application is the second function, the communication library can intercept the first function to execute the logic of the custom second function. This allows for the registration of a communication domain of a first capacity based on the amount of data exchanged multiple times during the information exchange phase.

[0011] Understandably, in traditional technology, each application sends an interface call request to the communication library to register a memory domain. The communication library then registers a corresponding MR (Memory Map) for each application. Registering MRs not only consumes system resources and has low efficiency, but the multiple registered MRs may not be contiguous, leading to numerous memory fragments. This application, during the information exchange phase, pre-registers a large first memory domain, then divides it into multiple blocks, and allocates blocks to each application. This allows a single contiguous memory domain to be used by multiple applications, reducing the number of memory fragments. It also reduces the number of memory domain registrations.

[0012] Furthermore, if there are too many memory domains, the network card's memory mapping table may not be able to store all the mapping relationships between MRs and application memory. Addresses without stored mapping relationships will experience cache misses, affecting the overall performance of the network card. The method of pre-registering memory domains and then dividing them into multiple blocks for use by multiple applications, as described in this application, can significantly reduce the number of MRs and reduce the performance loss caused by MTT cache misses.

[0013] In one possible implementation, the exchanged information of the second node also includes a key to the second node's memory domain. This key is obtained when the second node registers its memory domain and is used by other nodes besides the second node to access the second node's memory domain via RDMA communication.

[0014] In the specific implementation, the communication library can also send the first key of the first memory domain to the RDMA hardware for storage. After receiving the address information and the first key of the memory block, the RDMA hardware can determine the mapping relationship between the address information of the memory block and the corresponding physical address. Then, it pins the physical address to prevent it from being mapped to other virtual addresses. This ensures that the RDMA hardware can access the corresponding physical memory through the address information of each memory block without address changes due to the operating system's memory management.

[0015] Furthermore, the first key may include a first local key and a first remote key. The RDMA hardware of the first node can access the first memory domain through the first local key, and the RDMA hardware of the second node can access the first memory domain through the first remote key, reading or writing data from the first memory domain. The local key needs to be stored locally for use by the local RDMA hardware, while the remote key needs to be sent to other nodes acting as senders for use by their RDMA hardware to implement remote RDMA read / write operations.

[0016] Understandably, in traditional technologies, each RDMA operation registers a new memory domain and generates a new key. Therefore, a handshake message needs to be sent before each RDMA write or read to obtain the receiver's key, which significantly impacts communication efficiency in frequent point-to-point communication scenarios. However, in the technical solution of this application, after registering the first memory domain, dividing the memory blocks, obtaining the receiver's exchange information, and determining the receiver's memory block address and first key during the information exchange phase, in the communication phase, regardless of the number of point-to-point communications, it is no longer necessary to obtain the other party's key or send handshake messages. Furthermore, other applications also use the first memory domain for RDMA, so communication between other applications does not require sending keys. Data communication between multiple applications on two nodes only requires obtaining the key once, ensuring the security of RDMA memory access while reducing communication redundancy and improving communication efficiency.

[0017] In one possible implementation, the RDMA hardware includes a bus ordering function to record the data reception order. The method further includes the following steps: each time the first node completes a data communication, it sends a Transmission Acknowledgment (ATP) message to the second node. The ATP message is used to notify the second node that the data communication operation has ended. The first node receives multiple Acknowledgment (ACK) messages sent by the second node. The ACK messages are used to notify the first node that data reception or data writing is complete. The multiple ACK messages are sent by the second node according to the data communication order based on the bus ordering function.

[0018] In specific implementation, the network interface card with the aforementioned bus ordering feature refers to a network interface card that can guarantee the order of data transmission, ensuring that data packets can be processed in the order of sending or receiving, and avoiding out-of-order situations. This type of network interface card can be a Kunpeng network interface card or a network interface card from other manufacturers, and this application does not make any specific limitations.

[0019] It should be noted that after the first node receives the ACK message, if the RDMA operation fails to write, the ACK message will carry the information of the write failure. Although the first node has already sent the ATP message in advance, it can still re-execute the RDMA operation based on the ACK message carrying the information of the write failure.

[0020] Understandably, in traditional technologies, after the first node writes data to the second node via RDMA, it needs to wait for the second node to send an ACK message before the first node sends an ATP message to the second node. Only then is the RDMA operation considered complete, and the first node can proceed to the next RDMA operation. For frequent point-to-point communication, the first node needs to wait for an ACK message after each write operation, significantly impacting communication efficiency. However, the technical solution in this application uses RDMA hardware with bus-preserving functionality. After each RDMA operation is completed, the first node can send an ATP message at the software level to confirm the completion of the RDMA and directly begin the next RDMA operation. This avoids the waiting time required for the second node to send an ACK message after each RDMA operation, greatly saving communication time and improving efficiency for frequent point-to-point communication.

[0021] In one possible implementation, the address information of a memory block includes the starting address of the memory domain to which the memory block belongs, and the offset of the memory block based on the starting address. The offset refers to the starting position of the block within the entire first memory domain. Each block can be understood as a sub-region of the first memory domain, and the specific location of each block can be quickly determined using the aforementioned offset.

[0022] For example, the address information of block1 includes virtAddr + offset(0), the address information of block2 includes virtAddr + offset(B1), and so on, with the address information of blockn including virtAddr + offset(Bn-1). Here, virtAddr refers to the starting address of the first memory domain, and offset(i) refers to the offset of the block based on the starting address. The above examples are for illustration only, and this application does not impose any specific limitations.

[0023] Optionally, the length of each block can be the same or different. Under the same conditions, such as each memory block having a length of L, the address information of each memory block can include the starting address and offset of the first memory domain. The communication library can obtain the complete address range of each memory block based on the uniform length L. Under different conditions, such as 100 blocks of length L or 100 blocks of length 2L to meet different application requirements, the address information of each memory block can also include the length of the memory block. The communication library can obtain the complete address range of each block based on its offset, length, and the starting address of the first memory domain.

[0024] The above implementation, which records the address information of memory blocks based on the starting address and offset, makes the address information of each block more concise, reduces the amount of data exchanged, and improves the data communication efficiency during the information exchange phase, thereby improving the overall communication efficiency. Furthermore, if the size of each memory block is the same, the address information of each block exhibits a certain regularity. The address of each block can be obtained using the starting address and the block length, eliminating the need to send the address of each block to the second node. The second node can determine the address of a block based on its length, starting address, and block number, further reducing the amount of data required for information exchange.

[0025] In one possible implementation, the communication system is used to implement ensemble communication tasks in the distributed training of artificial intelligence (AI) models. These ensemble communication tasks include one or more of the following: reducescatter, allreduce, allgather, and all-to-all.

[0026] Understandably, the technical solution provided in this application is suitable for application scenarios that require frequent point-to-point communication based on RDMA technology, such as distributed training scenarios for AI models. During distributed training, each node is responsible for updating a portion of the model's parameters, but the goal of training is to train a complete model. Therefore, nodes need to communicate with each other to synchronize their trained data to other nodes through aggregate communication. Aggregate communication is a common communication method in distributed systems, and it is a global communication operation in which all nodes in the system participate. Typically, aggregate communication can include Allgather communication, Allreduce communication, Reducescatter communication, all-to-all communication, and many other types, which will not be listed here. During these aggregate communication processes, nodes frequently communicate point-to-point based on RDMA technology. Using the technical solution in this application can save the communication preparation operations for each RDMA communication, improve communication efficiency, and thus improve the training efficiency of the model.

[0027] In one possible implementation, the RDMA transmission mode may include the Bcopy transmission mode. In this mode, in conventional technology, each RDMA communication requires the sending end to copy the data to a first buffer, and then write the data from the first buffer to a second buffer using RDMA technology. The receiving end needs to copy the data from the second buffer to the application memory. In other words, each RDMA communication requires copying the data twice.

[0028] In this embodiment of the application, neither the sending end nor the receiving end needs to copy data in each RDAM communication. After the first node and the second node have completed S550A and S550B and exchanged information, the data can be directly written to the second node through RDMA technology in Bcopy mode. Each RDMA can be directly written without copying data, which not only omits the transmission preparation operation, but also the copy operation after transmission, thus improving the data transmission efficiency in Bcopy mode.

[0029] In one possible implementation, the RDMA transmission mode may include the Zcopy transmission mode. In this mode, in traditional technology, for each RDMA communication, the sending end needs to register the MR first, and then write the data from the registered MR to the receiving end's buffer through RDMA technology. The receiving end needs to copy the data from the buffer to the application memory. In other words, each RDMA communication requires registering the MR and copying the data once.

[0030] In this embodiment, neither the sending nor receiving end needs to register a memory register (MR) nor copy data during each RDMA communication. After the first and second nodes have exchanged information, in Zcopy mode, data can be directly written to the second node using RDMA technology, and then the first memory block can be unregistered. Each RDMA communication does not require MR registration or data copying, thus omitting both transmission preparation and post-transmission copying operations, improving data transmission efficiency in Zcopy mode.

[0031] In one possible implementation, the RDMA transmission mode may include the RNDV transmission mode. In this mode, in traditional technology, for each RDMA communication, the sender and receiver need to register their respective MRs first, then exchange MR information through handshake messages, and then the receiver sends an RTR message to the sender to request the start of data transmission. Only then will the sender start the RDMA operation. In other words, the transmission preparation operations before each RDMA communication include both parties registering MRs and exchanging handshake messages.

[0032] In this embodiment of the application, for each RDMA communication, the sending end and the receiving end do not need to register MR or exchange handshake messages. The sending end can directly perform RDMA operation, so the preparation operation before transmission can be completely omitted. There is no need to register MR or exchange handshake messages. Each RDMA saves the preparation operation before transmission. For frequent point-to-point transmission scenarios, it can greatly improve transmission efficiency.

[0033] Secondly, a computing node is provided, comprising: a switching unit for acquiring switching information of a second node during the information switching phase, the switching information including address information of memory blocks, the memory blocks being obtained by partitioning the memory domain of the second node, the memory domain of the second node being used by remote direct memory access (RDMA) hardware; and a data communication unit for communicating multiple times with the second node through the RDMA hardware of the computing node during the communication phase, each communication using the switching information to read or write data from the memory blocks.

[0034] Implementing the computing node described in the second aspect, during the information exchange phase, the computing node pre-registers a first memory domain with the RDMA hardware, enabling the RDMA hardware to directly access the first memory domain via RDMA technology. The first memory domain is then divided into multiple memory blocks, and the address information of each block is recorded. Thus, when the computing node and the second node have communication needs, they can exchange the address information of their respective memory domain blocks before communication. During the communication phase, when the computing node and the second node begin RDMA communication, each communication can directly perform RDMA read and write operations based on the address information of the blocks in the second node's memory domain. Each communication does not require transmission preparation operations, i.e., no data copying, no memory domain registration, and no handshake message sending, significantly improving communication efficiency in frequent point-to-point communication scenarios.

[0035] In one possible implementation, a registration unit is used to register a first memory domain of a first capacity to the RDMA hardware of the computing node. The first memory domain is used for RDMA communication by the RDMA hardware of the first node. The first capacity is not less than the data size of a single RDMA communication by the first node. A partitioning unit is used to divide the first memory domain into multiple memory blocks and obtain the address information of each memory block. A switching unit is used to generate switching information of the computing node based on the address information of each memory block. The switching information of the computing node is used for multiple communications between the third node and the computing node through the RDMA hardware.

[0036] In one possible implementation, the computing node includes a first application and a first communication library, the second node includes a second application, and there is a communication requirement between the first application and the second application. A registration unit is used to send a request to the first communication library to register a memory domain. The registration unit is also used to allocate at least one memory block from multiple memory blocks for the first application to use, based on the request for registering the memory domain, through the first communication library. Finally, the registration unit is used to generate exchange information for the computing node based on the address information of at least one memory block, through the first communication library. The exchange information for the computing node includes at least the address information of at least one memory block of the first application.

[0037] In one possible implementation, the exchanged information of the second node also includes a key to the second node's memory domain. This key is obtained when the second node registers its memory domain and is used by other nodes besides the second node to remotely access the second node's memory domain via RDMA technology.

[0038] In one possible implementation, the RDMA hardware includes a bus ordering function to record the data reception order. The node also includes a message sending unit and a message receiving unit. The message sending unit sends a Transmission Acknowledgment (ATP) message to the second node after each data communication is completed. The ATP message is used to notify the receiving end that the data transmission operation has ended. The message receiving unit receives multiple Acknowledgment (ACK) messages sent by the second node. The ACK messages are used to notify the sending end that data reception is complete. The multiple ACK messages are sent by the second node according to the data communication order based on the bus ordering function.

[0039] In one possible implementation, the address information of the memory block includes the starting address of the memory domain to which the memory block belongs, and the offset of the memory block based on the starting address.

[0040] In one possible implementation, the communication system is used to implement ensemble communication tasks in the distributed training of artificial intelligence (AI) models. These ensemble communication tasks include one or more of the following: reducescatter, allreduce, allgather, and all-to-all.

[0041] Thirdly, this application provides a distributed system comprising multiple computing nodes, each computing node being capable of implementing the operational steps implemented by the first node in the method of the first aspect or any possible implementation of the first aspect.

[0042] Fourthly, this application provides a computing device, which includes a processor, a memory, a communication interface, and a bus. The processor, the memory, and the communication interface are connected via the bus and communicate with each other. The memory stores computer execution instructions. When the processor is running, the processor executes the computer execution instructions in the memory to perform the operation steps of the method in the first aspect or any possible implementation of the first aspect using the hardware resources in the processor.

[0043] Fifthly, this application provides a computing device cluster, which includes multiple computing devices. Each computing device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected to each other via the bus and communicate with each other. The memory is used to store computer execution instructions. When the processor is running, the processor executes the computer execution instructions in the memory to use the hardware resources in the processor to perform the operation steps of the method in the first aspect or any possible implementation of the first aspect.

[0044] Sixthly, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0045] In a seventh aspect, this application provides a computer program product containing instructions that, when run on a computer, causes the computer to perform the methods described above.

[0046] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the RDMA write process in Bcopy mode;

[0048] Figure 2 This is a schematic diagram of the RDMA write process in Zcopy mode;

[0049] Figure 3 This is a schematic diagram of the RDMA write process in RNDV mode;

[0050] Figure 4 This is an architecture diagram of a communication system provided in this application;

[0051] Figure 5 This is an example diagram of a communication system 2000 deployed in a cloud environment according to this application;

[0052] Figure 6 This is a flowchart illustrating the steps of a data communication method provided in this application;

[0053] Figure 7 This is a flowchart illustrating the steps of a data communication method provided in this application under different transmission modes;

[0054] Figure 8 This is a flowchart illustrating the steps of the data communication method provided in this application in an application scenario;

[0055] Figure 9 This is a flowchart illustrating the steps of the data communication method provided in this application in another application scenario;

[0056] Figure 10 This is a schematic diagram of the software structure of a computing node provided in this application;

[0057] Figure 11 This is a schematic diagram of the structure of a computing device provided in this application;

[0058] Figure 12 This is an example diagram of a computing device cluster provided in this application. Detailed Implementation

[0059] RDMA (Redirect Memory Access) technology is a technique that allows direct access from the memory of one computing device to the memory of another, without requiring the operating system or CPU of the computing device. This improves data transfer efficiency, reduces network latency and CPU load, and is widely used in various scenarios. Typically, RDMA offers several data transfer modes, each suitable for different application scenarios and requirements, such as bcopy mode, zcopy mode, and RNDV mode.

[0060] The following example illustrates several commonly used data transmission modes of RDMA, using the example of the sending end needing to write data from the first application memory to the receiving end's second application memory.

[0061] Figure 1 This is a schematic diagram of the RDMA write process in Bcopy mode, as shown below. Figure 1 As shown, in Bcopy mode, the sending end copies the data to be sent from the first application memory to the first buffer of the sending end. The buffer is usually a pre-allocated buffer storage area in the communication library. Then, the data in the first buffer is written to the second buffer of the receiving end through RDMA. The receiving end then copies the data from the second buffer to the second application memory.

[0062] In this method, each RDMA transmission preparation operation includes a data copy operation at the sending end. Furthermore, after the transmission is complete, the receiving end also needs to perform a data copy operation, meaning that each RDMA transmission requires two data copies.

[0063] Figure 2 This is a schematic diagram of the RDMA write process in Zcopy mode, as shown below. Figure 2 As shown, in Zcopy mode, the sending end first registers the application memory where the data is located as a memory region (MR). This memory region is registered with the network interface card (NIC), allowing the NIC to directly access the MR. The sending end can directly write the data in the MR to the second buffer of the receiving end through RDMA operation. The receiving end then copies the data from the second buffer to the second application memory.

[0064] In this method, each RDMA transmission preparation operation includes the MR registration operation at the sending end, and after the transmission is completed, the receiving end also needs to perform a data copy operation. That is to say, each RDMA requires one MR registration and one data copy.

[0065] Figure 3 This is a schematic diagram of the RDMA write process in RNDV mode, as shown below. Figure 3 As shown, in RNDV mode, the following steps are included:

[0066] Step 1: The sending end registers the first application memory as the first MR, allowing Nic to access it directly; the receiving end registers the second application memory as the second MR, allowing Nic to access it directly.

[0067] Step 2: The sending end sends a handshake message carrying the first MR information to the receiving end, and the receiving end sends a handshake message carrying the second MR information to the sending end.

[0068] Step 3: The receiving end sends a Ready to Receive (RTR) message to the sending end. The RTR message is used to indicate that the receiving end is ready to receive data from the sending end and to inform the sending end that data transmission can begin.

[0069] Step 4: The sending end writes data from the first MR to the second MR using RDMA technology.

[0070] Step 5: The receiving end sends an Acknowledgement (ACK) message to the sending end. The ACK message is used to indicate that the receiving end has successfully received the data and to inform the sending end that the data has been successfully transmitted.

[0071] Step 6: The sending end sends a transmission acknowledgment to put (ATP) message to the receiving end. The ATP message is used to indicate that the sending end has completed the data transmission operation, the data transmission was successful and has ended, and to ensure that both parties have confirmed the success of the data transmission.

[0072] Step 7: The sending end releases the first MR, and the receiving end releases the second MR. This can also be called the MR cancellation operation.

[0073] In this mode, each RDMA transmission preparation operation includes the MR registration operation at the sending end, the MR registration operation at the receiving end, and the exchange of handshake messages between the two parties. That is to say, each RDMA requires one MR registration and one handshake message exchange.

[0074] The above text passed Figures 1-3 This section provides a detailed explanation of the remote write operation flowcharts for the three current RDMA transmission modes, illustrating the process of the sender writing data to the receiver. RDMA also supports remote read operations, where the receiver reads data from the sender. The process is similar to... Figures 1-3 It's similar, only the data flow changes, which won't be elaborated on here. Of course, RDMA has many other types of transmission modes, which will not be listed here.

[0075] based on Figures 1-3 It is known that regardless of the transmission mode, RDMA requires transmission preparation before any RDMA operation can be performed. Figure 1In the Bcopy mode shown, the transfer preparation operation refers to copying data from the first application memory to the first buffer. Figure 2 In the Zcopy mode shown, the transfer preparation operation refers to registering the MR operation. Figure 3 In the RNDV mode shown, the transmission preparation operation refers to the registration of the MR operation and the exchange of handshake messages. Transmission preparation operations are also required in other transmission modes. This is because RDMA bypasses the operating system or CPU to perform data read and write operations. Therefore, before RDMA operations, transmission preparation operations are needed to determine which memory regions can be accessed. Since the destination address for each data write is different, even in point-to-point communication scenarios where the sender and receiver remain constant, the memory address for each data write may still be different. Therefore, transmission preparation operations are required for each RDMA operation. In frequent communication scenarios, frequent transmission preparation operations can affect communication efficiency and increase communication latency.

[0076] For example, in Bcopy mode, if the sender and receiver frequently perform RDMA operations, each RDMA operation requires copying the data to the first buffer first. Frequent copying not only consumes the computing resources of the sender and receiver, but also affects the efficiency of RDMA writing.

[0077] In Zcopy mode, if the sender and receiver frequently perform RDMA operations, each RDMA operation requires MR registration. Frequent MR registration not only increases network latency (registering a 1MB MR typically takes 1ms) and affects RDMA write efficiency, but also impacts network card performance due to the large number of MRs. This is because as the number of MRs increases, the network card's memory translation table (MTT) may not be able to store all the mapping relationships between MRs and application memory. Addresses without stored mapping relationships will experience cache misses. Whenever a cache miss occurs, the network card needs to search for the required mapping information from main memory, thereby increasing transmission latency and affecting the overall performance of the network card.

[0078] In RNDV mode, if the sender and receiver frequently perform RDMA operations, each RDMA operation requires both parties to exchange handshake messages and register MRs. Frequent sending of handshake messages will consume network resources and affect the efficiency of data communication. Frequent MR registration will affect network card performance and increase transmission latency.

[0079] In summary, RDMA technology requires transmission preparation operations before each data transmission, such as registering memory regions, copying data in buffers, or exchanging handshake messages. When communication between the sending and receiving ends is frequent, these frequent transmission preparation operations will affect communication efficiency and increase communication latency.

[0080] To address the issue of RDMA's frequent transmission preparation operations impacting communication efficiency in scenarios with high communication frequency, this application provides a data communication method. This method includes an information exchange phase and a communication phase. In the information exchange phase, a first node pre-registers a first memory domain with the RDMA hardware, enabling the RDMA hardware to directly access the first memory domain via RDMA technology. The first memory domain is then divided into multiple memory blocks, and the address information of each block is recorded. Thus, when the first node and the second node have communication needs, they can exchange the address information of their respective memory domain blocks before communication. In the communication phase, when the first node and the second node begin RDMA communication, each communication can directly perform RDMA read and write operations based on the address information of the blocks in the second node's memory domain. Each communication does not require transmission preparation operations, i.e., no data copying, no memory domain registration, and no handshake message sending, significantly improving communication efficiency in frequent point-to-point communication scenarios.

[0081] Figure 4 This is an architecture diagram of a communication system provided in this application, such as... Figure 4 As shown, the architecture may include a client 1000 and a communication system 2000. A communication connection is established between the client 1000 and the communication system 2000. The communication connection may be a wired connection or a wireless connection. Wired connections include Ethernet, wired lines, cables, etc., while wireless connections include wireless local area networks (Wi-Fi), cellular networks, etc. It may also include the Internet, local area networks (LANs), etc., which support both wired and wireless connections. Of course, other connection types may also be included, but this application does not specifically limit them.

[0082] Client 1000 is deployed on terminal devices, computing devices, or edge computing devices. Terminal devices include personal computers, smartphones, wearable devices, handheld processors, tablets, mobile laptops, augmented reality (AR) devices, virtual reality (VR) devices, smart conferencing devices, etc., without specific limitations. Computing devices can be bare metal servers (BMS), virtual machines, or containers. BMS refers to a general-purpose physical server, such as an ARM server or an x86 server; a virtual machine refers to a complete computer system simulated by software, possessing full hardware system functionality and running in a completely isolated environment. Any task that can be performed on a physical computer can also be performed in a virtual machine. When creating a virtual machine on a computing device, a portion of the physical machine's hard drive and memory capacity needs to be used as the virtual machine's hard drive and memory capacity. Each virtual machine has its own independent basic input / output system (BIOS), hard drive, and operating system, allowing it to be operated like a physical machine. A container is a portable software unit that combines an application and all its dependencies into a single software package. This package is not limited by the underlying host operating system, eliminating the need to build complex environments and simplifying the application development and deployment process. Edge computing devices are hardware devices located close to the data source or data user.

[0083] The communication system 2000 includes multiple nodes, and these nodes can transmit data between each other using RDMA technology. Figure 4 Taking the first node 100 and the second node 200 as examples, the first node 100 is used to send data to the second node 200; that is, the first node 100 is the sender and the second node 200 is the receiver. Of course, the communication system 2000 may include more nodes, and the data communication method between other nodes can refer to the data communication method between the first node 100 and the second node 200. This application does not make specific limitations.

[0084] The first node 100 and the second node 200 can be computing devices, storage devices, or chips. The description of computing devices is as described above and will not be repeated here. Storage devices can specifically be hard disk drives (HDDs), solid-state drives (SSDs), mechanical hard disks (HDDs), USB flash drives (universal serial buses, USB), flash memory, SD cards (secure digital memory cards, SD cards), Memory Sticks, etc., and this application does not impose specific limitations. Storage arrays can be redundant arrays of independent disks (RAID), network attached storage (NAS), storage area networks (SANs), etc., and this application does not impose specific limitations. Chips can specifically be processors implemented using a central processing unit (CPU), or application-specific integrated circuits (ASICs), or programmable logic devices (PLDs). The aforementioned PLD can be implemented as a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system-on-chip (SoC), or any combination thereof, without specific limitation in this application. It may also include processors or accelerator cards that exhibit good performance when performing machine learning tasks, thereby improving the speed and efficiency of neural network training, such as graphics processing units (GPUs), neural network processing units (NPUs), data processing units (DPUs), tensor processing units (TPUs), etc., without specific limitation in this application.

[0085] The first node 100 and the second node 200 can also be a computing device cluster composed of multiple computing devices, a chip system composed of multiple chips, or a storage array composed of multiple storage devices. The storage array can be a redundant array of independent disks (RAID), network attached storage (NAS), storage area network (SAN), etc., and this application does not make specific limitations.

[0086] Optionally, client 1000 can be a client specifically for data storage, such as a database client, a distributed storage system client, or a file storage system client, etc., and this application does not impose any specific limitations. Client 1000 can receive data write requests sent by users, and then send an RDMA request to the first node 100. The first node 100 then sends the data to the second node 200 via RDMA technology using the data communication method provided in this application.

[0087] Optionally, client 1000 can also be a client that includes data storage functionality. Such clients may also include other functions, such as storage, computation, and task scheduling. For example, client 1000 can be a scheduling client for a high-performance computing (HPC) system. This client can receive user scheduling requests and schedule computing tasks to the first node 100. When the first node 100 executes a computing task, if it needs to send data to the second node 200, the first node 100 can send the data to the second node 200 via RDMA technology using the data communication method provided in this application. The above examples are for illustrative purposes; client 1000 can also be a big data processing client, an AI training platform client, etc., and this application does not impose specific limitations.

[0088] Optionally, client 1000 can also be a client of a cloud platform, allowing users to purchase and rent various cloud services. The data communication method provided in this application can be one of these cloud services, which users can purchase separately to achieve the purpose of the first node 100 writing data to the second node 200 via RDMA technology. Alternatively, the cloud platform provides a comprehensive service to users, and the data communication method provided in this application can be a sub-service of this comprehensive cloud service. For example, if the comprehensive cloud service is an AI cloud service, the data communication method provided in this application can be a sub-service of this AI cloud service, used to achieve aggregate communication during distributed training. During AI model training, if the first node 100 needs to perform aggregate communication with the second node 200, the data communication method provided in this application can be used to achieve the purpose of the first node 100 writing data to the second node 200 via RDMA technology.

[0089] The preceding text has described in detail the possible deployment methods for Client 1000 and Communication System 2000. In actual deployment, flexible deployment can be carried out based on specific application scenarios and business requirements. The following section provides examples of actual deployment methods for Client 1000 and Communication System 2000 using specific application scenarios.

[0090] As an example of an application scenario where the communication system 2000 is deployed on a local device, assuming that the client 1000 is a client of an enterprise database, enterprise employees can use the client to manage, query, maintain, store, and configure enterprise data. In this scenario, the client 1000 can be deployed on the enterprise's office computer, and the communication system 2000 is deployed on a server cluster purchased by the enterprise. The nodes in the communication system 2000 are the servers in this server cluster. Enterprise employees are users and can use their office computers to run the client 1000 to initiate a data storage request to store data A in the first node 100 in the second node 200. The first node 100 can receive the data storage request sent by the client 1000 and write data A into the second node 200 in RDMA mode using the data communication method of this application.

[0091] As an example of an application scenario where a communication system 2000 is deployed on a cloud platform, Figure 5 This is an example diagram of a communication system 2000 deployed in a cloud environment according to this application, such as... Figure 5As shown, a user can initiate a purchase request for AI cloud services through client 1000. After client 1000 sends the purchase request to the cloud platform, the cloud platform can provide cloud service access rights of communication system 2000 to client 1000, enabling the user to initiate a data storage request through client 1000. Client 1000 then sends the data storage request to communication system 2000, causing the first node 100 in communication system 2000 to write data A to the second node 200 using the data communication method of this application in the form of RDMA.

[0092] The cloud platform also maintains various basic resources, including computing resources, storage resources, network resources, and security resources, to meet the computing needs of the communication system 2000 under different scales and loads. Furthermore, these computing resources can be dynamically scaled according to the usage needs of the communication system 2000 to ensure the stable operation of the communication system 2000 and provide users with reliable RDMA services.

[0093] It should be understood that the above application scenarios are for illustrative purposes only. The client 1000 and communication system 2000 can be flexibly deployed according to actual business needs. They will not be listed here.

[0094] The possible system architecture of this application has been described above. The following is a brief explanation of how this system architecture implements the data communication method provided in this application.

[0095] In this embodiment, each node may include an application, a communication library, memory, and a network interface card (NIC). Taking the first node 100 as an example, the first node 100 includes a first memory 110, a first NIC 120, a first communication library 130, and a first application 140. The system may include an information exchange phase and a communication phase, which will be explained below.

[0096] During the information exchange phase, the first communication library 130 of the first node 100 can register a memory domain (MR) in the first memory 110 to obtain the memory domain of the first node, referred to as the first memory domain. This first memory domain is used for RDMA operations by the RDMA hardware of the first node or other remote nodes. The RDMA hardware may include, but is not limited to, hardware devices that support RDMA operations such as host channel adapters (HCAs). The registered first memory domain is a memory area that the RDMA hardware can directly access. Next, the first communication library 130 logically divides the first memory domain to obtain multiple memory blocks and records the address information of each first memory block. Each memory block is a logical memory block and has a mapping relationship with a physical memory block. Similarly, the second communication library 230 on the second node 200 can also register an MR in the second memory 210 to obtain the memory domain of the second node, referred to as the second memory domain, and then logically divide the second memory domain to obtain multiple memory blocks and record the address information of each second memory block.

[0097] The first application 140 can send a memory domain registration request to the first communication library 130. The first communication library 130 can select at least one first memory block from multiple memory blocks based on the memory domain registration request, which may include the memory size required by the first application 140. Figure 4 Block 1 in the first node 100 is assigned to the first application 140. Similarly, the second application 240 on the second node 200 can also send a memory domain registration request to the second communication library 230 in the same way. The second communication library 230 can select at least one second memory block from multiple memory blocks. Figure 4 The second node 200 (blcok1) is assigned to the second application 240.

[0098] When there is a communication requirement between the first application 140 and the second application 240, the first node 100 sends a first exchange information to the second node 200, and the second node 200 sends a second exchange information to the first node. The first exchange information includes at least the address information of the first memory block, and the second exchange information includes at least the address information of the second memory block.

[0099] During the communication phase, the first node 100 can remotely write data from the first memory block to the second memory block using RDMA technology based on the address information of the second memory block, without the intervention of the CPU. Furthermore, if the first application 140 needs to conduct multiple point-to-point communications with the second application 240, subsequent communications do not require sending handshake messages. During each communication, the first node 100 can remotely write data from the first memory block to the second memory block using RDMA technology based on the address information of the second memory block received previously.

[0100] Understandably, this application pre-registers a memory domain and then logically divides the memory domain to obtain multiple memory blocks. Each application can be allocated a corresponding memory block. When application processes on different nodes need to perform multiple point-to-point communications, the address information of the memory block allocated by the application can be sent to the other party in advance. This allows subsequent point-to-point communications to use the address information of the memory block to complete RDMA writes without frequently sending handshake messages. This is because existing technologies require sending handshake messages because the specific address written in each point-to-point communication is different. However, this application pre-exchanges the address information of at least one memory block. Since the address information of a memory block can be understood as a storage range, after pre-exchanging the address information of the memory blocks, multiple data storages can be performed within this storage range without frequently sending handshake messages to determine the write address. This avoids the resource waste caused by the need for transmission preparation operations, such as sending handshake messages, for each communication when the sender and receiver communicate frequently.

[0101] Optionally, the address information of the memory blocks in the aforementioned exchange information may include the starting address of the memory domain where the memory block is located and the offset of each memory block's address relative to the starting address. In a specific implementation, the aforementioned first address information includes the starting address of the first memory domain and the offset of each first memory block based on the starting address, and the aforementioned second address information includes the starting address of the second memory domain and the offset of each second memory block based on the starting address.

[0102] Optionally, the exchanged information may also include the key information of the memory domain. Specifically, after the first node 100 completes the registration of the first memory domain, it can obtain the first key (MR key) of the first memory domain. Similarly, after the second node 200 completes the registration of the second memory domain, it can obtain the second key of the second memory domain. When the first node 100 and the second node 200 send exchange information to each other, they not only exchange the address information of the memory blocks, but also exchange the key of the memory blocks, so that the other party can use the key to remotely access its own memory blocks.

[0103] Thus, in the bcopy scenario, Figure 1In the bcopy technology shown, each time data communication occurs, the sending end needs to first copy the data to a buffer prepared in advance by the communication library, and then remotely write the data from the sending end's buffer to the receiving end's buffer via RDMA. The receiving end then copies the data from the buffer to the application memory. Therefore, the traditional bcopy technology requires two copies for each RDMA communication, which not only affects communication efficiency but also consumes a lot of system resources. Figure 4 In the technical solution of this application shown, data does not need to be copied for each RDMA communication, which can improve communication efficiency and reduce system resource consumption.

[0104] In the zcopy scenario Figure 2 In the zcopy technology shown, each time data communication occurs, the sending end needs to register a memory domain for RDMA hardware access. Then, the data in the memory domain is written to a pre-prepared buffer in the receiving end's communication library via RDMA remote writing. The receiving end then copies the data from the buffer to the application memory. Therefore, the traditional zcopy technology requires one memory domain registration and one copy for each RDMA communication, which not only affects communication efficiency but also consumes a large amount of system resources. Figure 4 In the technical solution of this application shown, data does not need to be copied for each RDMA communication. The memory domain is pre-registered, and the memory domain does not need to be registered for each communication, which can improve communication efficiency and reduce system resource consumption.

[0105] In the RNDV scenario Figure 3 In the RNDV technology shown, each time data communication occurs, the sending and receiving ends need to register memory domains for RDMA hardware access. Then, they exchange handshake messages to obtain the source and destination addresses for this RDMA communication. The sending end then writes the data from its memory domain to the receiving end's memory domain via RDMA remote writing. Therefore, traditional RNDV technology requires both parties to register memory domains and exchange handshake messages for each RDMA communication, which not only affects communication efficiency but also consumes significant system resources. Figure 4 In the technical solution of this application shown, the memory domain is pre-registered, the memory domain is divided into multiple memory blocks, and the corresponding number of memory blocks are allocated to the application. In this way, after the two communicating parties exchange the address information of the memory blocks, each communication does not need to register the memory domain or send a handshake message, which can improve communication efficiency and reduce system resource consumption.

[0106] Furthermore, in the RNDV scenario, such as Figure 3In the steps described in the embodiment, traditional RDMA technology, after the sending end writes data to the receiving end, needs to wait for the receiving end to send an ACK message before sending an ATP message. An ACK message is an acknowledgment message for the completion of a specific RDMA operation; the receiving end sends an ACK message to the sending end to indicate that data reception is complete. An ATP message is sent by the sending end to the receiving end to confirm that data transmission has been successfully completed, indicating that the sending end has received the ACK message and the RDMA operation is confirmed as complete. Based on the definitions of ACK and ATP messages, it is clear that after each RDMA operation ends, the sending end needs to wait for an ACK message from the receiving end to confirm that the transmission has successfully ended. An ATP message will not be sent if no ACK message is received from the receiving end, and the time spent waiting for the ACK message is also included in the total operation time of the RDMA operation, leading to increased latency in the RDMA operation.

[0107] In this embodiment, using a network interface card with bus ordering feature, after the first node 100 writes data RDMA to the second node 200, it can directly send an ATP message to the second node 200 without waiting for the second node 200 to send an ACK message. The second node 200 can continuously send ACK messages to the first node 100 in the order of receipt, thereby further improving the communication efficiency of RDMA.

[0108] In specific implementation, the network interface card with the aforementioned bus ordering feature refers to a network interface card that can guarantee the order of data transmission, ensuring that data packets can be processed in the order of sending or receiving, and avoiding out-of-order situations. This type of network interface card can be a Kunpeng network interface card or a network interface card from other manufacturers, and this application does not make any specific limitations.

[0109] Understandably, after the first node receives the ACK message, if the write operation fails, the ACK message will carry information indicating the write failure. Although the first node has already sent an ATP message, it can still re-execute the RDMA operation based on the ACK message carrying the write failure information. In this way, by utilizing the network card with bus-preserving characteristics, after each RDMA operation is completed, the first node can send an ATP message to confirm the completion of RDMA at the software level, avoiding the waiting time required for the second node to send an ACK message for each RDMA operation. For frequent point-to-point communication, this can greatly save communication time and improve communication efficiency.

[0110] For example, in Figure 4 In the example shown, the first communication library 130 can first register the first memory domain in the first memory 110 to obtain the first memory domain ( Figure 4Taking MR1 as an example, the first application 140 stores the address and key information of each block, and then divides MR1 ​​into n blocks of equal size. After the first application 140 sends a memory domain registration request by calling the memory domain registration interface in the first communication library 130, the first communication library 130 can allocate blocks 11 and 12 to the first application 140 according to the memory size required by the first application 140. The first network interface card 120 can store the address information of each block and the key of the memory domain in which the block is located. Similarly, the second network interface card 220 can also store the address information of each block and the key of the memory domain in which the block is located.

[0111] Next, when the first application 140 and the second application 240 have a communication requirement, the first network interface card 120 sends a first exchange message to the second network interface card 220, including a first address information and a first key. The second network interface card 220 sends a second exchange message to the first network interface card 120, including a second address information and a second key. This allows the first network interface card 120 to use the second key to remotely access the second memory domain of the second node and write the data in block 11 into block 21 using RDMA technology. In each subsequent communication process, the second key can be used directly to access the second memory domain. Each communication does not require registering the memory domain or exchanging handshake messages, thus reducing the number of preparation operations and improving communication efficiency.

[0112] It's important to note that the address information stored by the network card is logical address information. The network card can also store the mapping between the logical address and physical address of each block to enable RDMA remote writes. It should be understood that applications typically do not interact directly with physical memory, but rather access it directly through virtual memory. The operating system and hardware translate virtual addresses into physical addresses. The logical address information refers to a memory region within that virtual memory space; therefore, the network card can also store the mapping between logical address information and physical address information.

[0113] Understandably, by pre-registering large memory domains in this way and then dividing them into multiple blocks for different applications, the number of memory domain registrations can be reduced. Each memory domain registration consumes certain system resources. In traditional technologies, a memory domain needs to be registered every time there is communication, which leads to resource waste. The technical solution of this application only needs to pre-register large memory domains based on the historical memory usage of multiple applications on the node. The size of the memory domain should be as close as possible to the usage needs of multiple applications, which can reduce the number of memory domain registrations and avoid resource waste.

[0114] In summary, the communication system provided in this application allows nodes to pre-register memory domains for direct access by RDMA hardware. The registered memory domains are then logically divided into multiple memory blocks. At least one memory block is allocated to each application based on its memory requirements. When applications on different nodes have communication needs, they can exchange memory block information before the first communication. In subsequent communications, the sending end can perform RDMA remote writing based on the other party's memory block information. No further transmission preparation is required for each communication; that is, no memory domain registration or handshake message sending is needed. This not only improves communication efficiency but also reduces the waste of system resources.

[0115] The communication system provided in this application has been described in detail above. The data communication method provided in this application will be explained below with reference to the accompanying drawings.

[0116] Figure 6 This is a flowchart illustrating the steps of a data communication method provided in this application, which can be applied to... Figure 4 The communication system shown. The data communication method may include an information exchange phase and a communication phase, wherein S510A to S550B describe the information exchange phase, and S560 describes the communication phase.

[0117] like Figure 6 As shown, the data communication method may include the following steps:

[0118] S510A: First node 100 registers the first memory domain to the RDMA hardware.

[0119] In practice, registering a memory domain refers to registering a local memory region to the RDMA hardware through the operating system or RDMA driver. The RDMA hardware may include hardware devices that support RDMA operation, such as network cards and host channel adapters (HCAs), so that the RDMA hardware can directly access the registered memory domain without the intervention of the CPU or operating system.

[0120] In the specific implementation, after registering the first memory domain, the first node 100 can obtain the starting address information of the first memory domain, the length of the first memory domain, and the first key. This first key can be used by remote nodes in RDMA operations to ensure the security of memory access. For example, if the first node 100 sends the first key to the third node, the third node can use the first key to remotely access the first memory domain.

[0121] It should be noted that the key obtained after registering the memory domain typically includes a remote key (RK) and a local key (LK). The RK can be used by the remote node in RDMA operations, while the LK is used to represent a specific local process, application, or resource to restrict the access range of the memory region. The aforementioned first key can be the RK used to send to the remote second node.

[0122] Optionally, the first memory domain can be used for multiple RDMA operations. Therefore, the capacity of the first memory domain registered by the first node 100 is a first capacity, which is at least greater than the data size of a single RDMA transfer. This allows a single registration to be used for multiple RDMA operations, reducing the number of memory domain registrations and thus saving system resources. In specific implementations, the first capacity can be determined based on the historical memory usage of multiple applications in the first node 100. Specifically, it can be set by the user or obtained by the system after analyzing historical data; this application does not impose specific limitations.

[0123] Optionally, the first node 100 can register its memory domain using a communication library. This communication library is a software library used to manage and optimize data exchange between different computing nodes or processes, providing developers with data transfer capabilities across processes, nodes, or networks. Communication libraries are typically used in distributed systems, high-performance computing, parallel computing, data centers, and cloud computing scenarios. The communication library can include multiple API functions to implement different operational logics, such as network communication, memory management, and data transfer, enabling developers to efficiently and easily implement inter-process communication or cross-node data transfer.

[0124] In traditional technologies, the size of the memory domain registered each time is determined by the size of a single RDMA data transfer. However, the size of the memory domain registered in this application is the first capacity. Therefore, assuming that the API function for registering a memory domain in the traditional way is the first function, and the API function for registering a memory domain in this application is the second function, the communication library can intercept the first function to make the communication library execute the logic of the custom second function. In this way, the size of the memory domain registered each time is much larger than the size of a single data transfer, so that multiple RDMAs can be implemented using the pre-registered memory domain, thereby avoiding the need to register MR for each RDMA and reducing the waste of system resources.

[0125] S510B: Second node 200 registers the second memory domain to the RDMA hardware.

[0126] In the specific implementation, the second node 200 registers the second memory domain in the same way as the first node 100 registers the first memory domain. The second node can also register the second memory domain in the same way to obtain the starting address information of the second memory domain, the length of the second memory domain, and the second key. For the specific process, please refer to the relevant description of S501A, which will not be repeated here.

[0127] S520A: The first node 100 divides the first memory domain into multiple memory blocks.

[0128] Alternatively, the operation of dividing memory into blocks can also be implemented using the aforementioned communication library. This library can logically divide the first memory domain into multiple blocks. Here, logical division refers to dividing the logical addresses (or virtual addresses) of the first memory domain, not the physical addresses. Logical addresses are the addresses used by the program at runtime; the program accesses memory through logical addresses, while the operating system is responsible for translating these logical addresses into their corresponding physical addresses. Physical addresses are the addresses within the actual memory chip; hardware uses physical addresses to locate the positions where data is stored.

[0129] Optionally, the communication library can divide the first memory domain according to a preset block length based on the length of the first memory domain and the starting address information of the first memory domain. The preset block length can be user-defined, a system default, or determined by the system based on the historical memory usage of multiple applications on the node; this application does not impose a specific limitation. In practice, larger blocks can reduce the management overhead of the communication library but may lead to more memory fragmentation, while smaller blocks can improve memory utilization but increase management overhead. Therefore, the block size can be flexibly defined based on the historical memory usage of multiple applications on the node; this application does not impose a limitation on this.

[0130] Optionally, after the communication library divides the memory into multiple blocks, it can record the address information of each block. This address information is used to indicate the address range of the block. In a specific implementation, the address information of each block may include the starting address of the first memory domain and the offset of each block. The offset refers to the starting position of the block in the entire first memory domain. Each block can be understood as a sub-region of the first memory domain. The specific location of each block can be quickly located using the offset.

[0131] For example, the address information of block1 includes virtAddr + offset(0), the address information of block2 includes virtAddr + offset(B1), and so on, with the address information of blockn including virtAddr + offset(Bn-1). Here, virtAddr refers to the starting address of the first memory domain, and offset(i) refers to the offset of the block based on the starting address. The above examples are for illustration only, and this application does not impose any specific limitations.

[0132] Optionally, the length of each block can be the same or different. In the case of the same length, such as each memory block having a length of L, the address information of each memory block can include the starting address of the first memory domain and its offset. The communication library can obtain the complete address range of each memory block based on the uniform length L. In different cases, such as 100 blocks of length L and 100 blocks of length 2L to meet different application requirements, the address information of each memory block can also include the length of the memory block. The communication library can obtain the complete address range of each block based on its offset, length, and the starting address of the first memory domain. The above examples are for illustration only and are not specifically limited in this application. For ease of understanding, the following explanation uses the example of blocks having the same length.

[0133] S520B: The second node 200 divides the second memory domain into multiple memory blocks.

[0134] In the specific implementation, the second node 200 divides the second memory domain in the same way as the first node 100 divides the first memory domain. After the second node 200 divides the memory into multiple memory blocks, it can also obtain the address information of each memory block. For a detailed description, please refer to S520A, which will not be repeated here.

[0135] S530A: The first node 100 allocates at least one first memory block for the first application.

[0136] In a specific implementation, the first application can send an interface call request for memory domain registration to the communication library. In response to the call request, the communication library allocates at least one first memory block to the first application from the already registered first memory domain. The interface call request may include the memory requirement of the first application, and the communication library can allocate a corresponding number of first memory blocks to the first application based on the memory requirement and the length of each memory block.

[0137] Furthermore, after the communication library allocates at least one first memory block for the first application, it can record the address information of the at least one first memory block corresponding to the first application, and then send the address information of the at least one first memory block to the first network interface card 120. The first network interface card 120 can record the mapping relationship between the address information of the at least one first memory block and the physical address in memory, so as to perform RDMA operations later. Similarly, other applications can also use the same method to send an interface call request for memory domain registration to the communication library. The communication library can determine the address information of the memory block corresponding to each application, and then send it to the first network interface card 120. The first network interface card records the mapping relationship between the address information and the physical address, so as to perform RDMA operations later.

[0138] Understandably, in traditional technology, each application sends an interface call request to the communication library to register a memory domain. The communication library then registers a corresponding MR for each application. Registering MRs not only consumes system resources and has low efficiency, but the registered MRs may not be contiguous, leading to numerous memory fragments. This application pre-registers a large first memory domain, then divides it into multiple blocks, and allocates blocks to each application. This allows a single contiguous memory domain to be used by multiple applications, reducing the number of memory fragments. It also reduces the number of registered memory domains. As mentioned earlier, if the number of memory domains is too large, the network card's MTT (Mean Transmission Terminal) may not be able to store the mapping relationship between all MRs and application memory. Addresses without stored mapping relationships will experience cache misses, affecting the overall performance of the network card. This application's method of pre-registering memory domains and then dividing them into multiple blocks for use by multiple applications can significantly reduce the number of MRs and reduce performance losses caused by MTT cache misses.

[0139] Furthermore, the communication library can send the address information of each memory block to the network card, and can also send the first key of the first memory domain to the network card for storage. After receiving the address information and the first key of the memory block, the network card can determine the mapping relationship between the address information of the memory block and the corresponding physical address, and then pin the physical address to prevent the physical address from being mapped to other virtual addresses. This ensures that the network card can access the corresponding physical memory through the address information of each memory block, and that the address will not change due to the memory management of the operating system.

[0140] S530B: The second node 200 allocates at least one second memory block for the second application.

[0141] In the specific implementation, the steps of the second node 200 to allocate the second memory block for the second application are the same as the steps of the first node 100 to allocate the first memory block for the first application. For details, please refer to the relevant description in S530A. It will not be repeated here.

[0142] S540A: The first node 100 obtains the first exchange information of the first application.

[0143] Optionally, the first exchange information may include a first key for the first memory domain, which is used by a remote node (such as the second node 200) to remotely access the first memory domain of the first node 100. A detailed description of the first key can be found in the relevant descriptions of S510A to S510B, and will not be repeated here.

[0144] Optionally, the first exchange information may include first address information, which includes at least the address information of at least one first memory block allocated for the first application, and the address information is used to indicate the address range of each first memory block. In a specific implementation, the first address information may include the starting address of the first memory domain and the offset of each first memory block. The description of the starting address and the offset can be found in the relevant descriptions of S520A to S520B, and will not be repeated here.

[0145] Furthermore, the first address information may also include the address information of all memory blocks in the first memory domain. That is, it includes not only the address information of the memory blocks allocated to the first application, but also the address information of the memory blocks allocated to other applications. In this way, when the two nodes communicate, all applications can complete RDMA based on the address information of the memory blocks in the first exchange information.

[0146] It should be noted that if S540A and S540B exchange the address information of all memory blocks, the first node 100 and the second node 200 can only know the address information of each other's memory blocks, but do not know which memory block corresponds to the application. Therefore, the aforementioned first address information may also include the memory block information corresponding to the application. Alternatively, the first node 100 and the second node 200 may send a notification message to each other before communication, informing each other of the memory block information corresponding to the application in this communication.

[0147] In simple terms, if the first exchange information only includes the address information of the memory block allocated to the first application, then the first and second applications can use this first exchange information to complete multiple point-to-point communications without sending handshake messages. However, if other applications of the first node 100 and the second node 200 need to communicate, such as the third application of the first node needing to communicate with the fourth application of the second node, then the first node needs to send new exchange information to the second node. But after sending the exchange information once, the third and fourth applications do not need to send handshake messages for point-to-point communication.

[0148] If the first exchange information includes the address information of all memory blocks in the first memory domain, as well as the information of the memory blocks allocated to each application, then the first application and the second application can use this first exchange information to complete multiple point-to-point communications, or other applications can use this first exchange information to complete multiple point-to-point communications, without sending handshake messages.

[0149] If the first exchange information only includes the address information of all memory blocks in the first memory domain, then the first application and the second application can send a synchronization message to each other before needing to communicate. This synchronization message includes the memory block information corresponding to the first and second applications, such as the memory block ID. In this way, the first node and the second node can determine the address information of the memory blocks allocated to the first and second applications to complete the RDMA operation. Subsequent point-to-point communication between the first application and the second application can be completed based on the above memory block address information without sending handshake messages. However, if other applications of the first node 100 and the second node 200 need to communicate, such as the third application of the first node needing to communicate with the fourth application of the second node, then the first node needs to send a new synchronization message to the second node. This communication message includes the memory block information allocated to the third and fourth applications, such as the memory block ID. However, after sending one synchronization message, the third application and the fourth application do not need to send handshake messages for point-to-point communication.

[0150] The above examples provide possible data for the exchange of information. In different scenarios, the data can be flexibly selected according to business needs. This application does not impose any specific limitations.

[0151] S540B: ​​The second node 200 obtains the second exchange information for the second application.

[0152] In a specific implementation, the second exchange information includes at least the second address information of at least one second memory block allocated for the second application, and the second key of the second memory domain. The description of the second exchange information is similar to that of the first exchange information. For details, please refer to the relevant description of S540A. It will not be repeated here.

[0153] S550A: First node 100 sends the first exchange information to second node 200.

[0154] Optionally, the first node 100 may send first exchange information to the second node 200 if it is determined that there is a communication need between the first node 100 and the second node 200. The specific sending time can be before the first node 100 sends data to the second node 200 for the first time. For example, if the first node and the second node need to jointly process a task and it is determined that there is a communication need between them, then they can send exchange information to each other before they begin processing the task; this application does not impose specific limitations.

[0155] For example, in a ensemble communication scenario during AI training, each node processes a portion of the training data and obtains gradient data through its own training. In order to update the model parameters, this gradient data needs to be exchanged between different nodes for gradient aggregation. During this process, there is a communication requirement between each node, which needs to synchronize the gradient data it has calculated. Therefore, before the AI ​​training task begins, the nodes that need to communicate can send and exchange information to each other so that after the training task begins, each node can complete RDMA communication based on the exchanged information.

[0156] For another example, in high-performance computing (HPC) scenarios, a large amount of numerical computation needs to be performed between nodes, and data exchange between nodes is frequent. In computationally intensive applications such as numerical simulation and weather forecasting, multiple nodes need to share a large amount of intermediate computation results. Therefore, before the computation task begins, the nodes that need to communicate can send and exchange information to each other so that after the computation task begins, each node can complete RDMA communication based on the exchange information.

[0157] The above examples illustrate application scenarios where communication is required between nodes, as well as the time required to send and exchange information in each application scenario. The examples are for illustration purposes only, and this application can also be applied to other scenarios where there is a need for RDMA data communication. This application does not make any specific limitations.

[0158] Optionally, S550A and S550B can occur simultaneously or sequentially. In the sequential case, the first node 100, acting as the sender, can execute S550A first, and the second node 200, acting as the receiver, can execute S550B after receiving the first exchange information and sending the second exchange information to the first node 100. This application does not impose specific limitations.

[0159] S550B: The second node 200 sends the second exchange information to the first node 100.

[0160] In the specific implementation, the steps of the second node 200 sending the second exchange information to the first node 100 are similar to the steps of the first node 100 sending the first exchange information to the second node 200. For details, please refer to the relevant description of S550A. It will not be repeated here.

[0161] It should be noted that in some application scenarios, if data transmission is unidirectional, such as when the first node writes data to the second node and the second node does not write data back to the first node, then the first node only needs to obtain the second exchange information from the second node, and the first node does not need to send the first exchange information to the second node. If data transmission is bidirectional, then the second node also needs to obtain the first exchange information from the first node, but this application does not impose specific limitations on this.

[0162] S560: The first node 100 completes multiple RDMA communications based on the first exchange information and the second exchange information.

[0163] In the specific implementation, each time the first node 100 sends data to the second node 200, it can be done based on the first exchange information and the second exchange information. This is because the first node 100 has already learned the address information of the second memory block and the second key. The address information of the second memory block is a storage range. The address written each time may be different, but it is within the storage range. Therefore, subsequent point-to-point communications do not need to send handshake messages, thereby avoiding resource waste and improving communication efficiency.

[0164] refer to Figures 1-3 As can be seen from the description of the embodiments, RDMA has multiple transmission modes, which will be discussed below. Figure 7 The specific processes of S560 under different transmission modes are explained. Figure 7 This is a flowchart illustrating the steps of a data communication method provided in this application under different transmission modes. Figure 7 S561 describes the RDMA process in Bcopy mode, S562-S563 describe the RDMA process in Zcopy mode, and S564-S568 describe the RDMA process in RNDV mode.

[0165] For Bcopy transmission mode, in traditional technologies, such as Figure 1 As shown, in each RDMA communication, the sending end needs to copy the data to the first buffer, and then write the data from the first buffer to the second buffer through RDMA technology. The receiving end needs to copy the data from the second buffer to the application memory. In other words, each RDMA communication requires copying the data twice.

[0166] In the embodiments of this application, such as Figure 7As shown, in each RDAM communication, neither the sending nor receiving end needs to copy data. After the first and second nodes have completed S550A and S550B and exchanged information, in Bcopy mode, data can be directly written to the second node through RDMA technology. Each RDMA can be written directly without copying data, which not only eliminates the need for transmission preparation operations but also eliminates the need for copy operations after transmission, thus improving the data transmission efficiency in Bcopy mode.

[0167] For Zcopy transfer mode, in traditional technologies, such as Figure 2 As shown, for each RDMA communication, the sending end needs to register the MR first, and then write the data from the registered MR to the receiving end's buffer using RDMA technology. The receiving end needs to copy the data from the buffer to the application memory. In other words, each RDMA communication requires registering the MR and copying the data once.

[0168] In the embodiments of this application, such as Figure 7 As shown, in each RDMA communication, neither the sending nor receiving end needs to register a memory register (MR) nor copy data. After the first and second nodes have completed the S550A and S550B operations and exchanged information, in Zcopy mode, data can be directly written to the second node via RDMA technology, and then the first memory block can be unregistered. Each RDMA communication does not require MR registration or data copying, thus eliminating both transmission preparation and post-transmission copying operations, improving data transmission efficiency in Zcopy mode.

[0169] For RNDV transmission mode, in traditional technologies, such as Figure 3 As shown, in each RDMA communication, the sending end and the receiving end need to first perform step 1 to register their respective MRs, then perform step 2 to exchange MR information through handshake messages, and then the receiving end performs step 3 to send an RTR message to the sending end to request the start of data transmission. Only then will the sending end start to perform step 4 to perform RDMA operation. In other words, the transmission preparation operations before each RDMA communication include both parties registering MRs and both parties exchanging handshake messages.

[0170] In the embodiments of this application, such as Figure 7As shown, in each RDMA communication, neither the sending nor receiving end needs to perform step 1 (register MR) nor step 2 (exchange handshake messages). They can directly perform steps 3 and 4. The receiving end sends an RTR message to request the start of data transmission, and the sending end performs the RDMA operation. This completely eliminates the need for pre-transmission preparation operations, as there is no need to register MR or exchange handshake messages. Each RDMA operation saves pre-transmission preparation operations, which can greatly improve transmission efficiency in frequent point-to-point transmission scenarios.

[0171] Meanwhile, after RDMA ends, traditional technology requires the sending end to wait for the receiving end to send an ACK message indicating acceptance before sending an ATP message indicating successful completion of the transmission. Only then will both parties unregister their respective memory blocks, and the RDMA operation ends. The delay caused by waiting for the ACK message affects the processing latency of the entire RDMA process. In scenarios with frequent point-to-point operations, this delay will affect the overall data transmission efficiency.

[0172] In the embodiments of this application, such as Figure 7 As shown, after each RDMA operation, the first node directly sends an ATP message without waiting for the second node to send an ACK message. Utilizing the network card with bus ordering feature, after the first node 100 writes the data to the second node 200 via RDMA, it can directly send an ATP message to the second node 200 without waiting for the second node 200 to send an ACK message. The second node 200 can continuously send ACK messages to the first node 100 in the order of reception, thereby further improving the communication efficiency of RDMA.

[0173] In specific implementation, the network interface card with the aforementioned bus ordering feature refers to a network interface card that can guarantee the order of data transmission, ensuring that data packets can be processed in the order of sending or receiving, and avoiding out-of-order situations. This type of network interface card can be a Kunpeng network interface card or a network interface card from other manufacturers, and this application does not make any specific limitations.

[0174] Understandably, after the first node receives the ACK message, if the write operation fails, the ACK message will carry information indicating the write failure. Although the first node has already sent an ATP message, it can still re-execute the RDMA operation based on the ACK message carrying the write failure information. In this way, by utilizing the network card with bus-preserving characteristics, after each RDMA operation is completed, the first node can send an ATP message to confirm the completion of RDMA at the software level, avoiding the waiting time required for the second node to send an ACK message for each RDMA operation. For frequent point-to-point communication, this can greatly save communication time and improve communication efficiency.

[0175] based on Figure 7As can be seen, when using the technical solution of this application for RDMA operation, in Bcopy transmission mode, neither the first node nor the second node needs to copy data during each RDMA operation; in Zopy transmission mode, neither the first node needs to register the MR, nor does the second node need to copy data during each RDMA operation; in RNDV mode, neither the first node nor the second node needs to register the MR, nor exchange handshake messages during each RDMA operation. The above explanation uses three RDMA transmission modes as examples. For the specific procedures of using the technical solution of this application in other transmission modes, please refer to the descriptions of the three transmission modes mentioned above; they will not be illustrated here.

[0176] It should be noted that for Bcopy and Zcopy modes, since there are no handshake messages or RTR messages in these two modes, after the first node as the sender completes the data copy or registers the MR, it does not know whether the second node as the receiver is ready to receive data. At this time, by analyzing the application's communication characteristics and memory access (memory access and memory storage) characteristics, it can be determined whether the application has frequent point-to-point communication memory access behavior. If not, RDMA operation can be started directly. If it does, it means that the application may need to read and write memory. A message can be sent to the receiver to confirm that the receiver is ready to receive data before starting data transmission, or RDMA operation can be started directly. If the operation fails, a prompt message should be sent to the user. This application does not make specific limitations on this.

[0177] Understandably, the technical solution of this application pre-registers a first memory domain of a first capacity, and then divides the first memory domain into multiple blocks, which are allocated to different applications. When the first node and the second node have communication needs, they exchange the address information of the blocks. This allows the first node and the second node to perform RDMA communication in the future based on the previously exchanged address information of the blocks, without needing to register the memory domain again or exchange handshake messages again. Each RDMA communication preparation operation is omitted, which can greatly save communication time and improve communication efficiency in frequent point-to-point communication scenarios.

[0178] The technical solution provided in this application is applicable to application scenarios requiring frequent point-to-point communication based on RDMA technology, such as distributed training scenarios for AI models. During distributed training, each node is responsible for updating a portion of the model's parameters, but the goal of training is to train a complete model. Therefore, nodes need to communicate with each other, synchronizing their trained data with other nodes through aggregate communication. Aggregate communication is a common communication method in distributed systems, involving global communication operations from all nodes within the system. Typically, aggregate communication includes Allgather, Allreduce, Reducescatter, and all-to-all communication, and may include many other types, which are not listed here. During these aggregate communication processes, nodes frequently engage in point-to-point communication based on RDMA technology. Using the technical solution in this application can save on communication preparation operations for each RDMA communication, improving communication efficiency and thus improving model training efficiency.

[0179] After Allgather communication ends, each node obtains the data from the other nodes. After Allreduce communication ends, each node obtains the sum of the data from all nodes. After ReduceScatter communication ends, each node obtains a portion of the sum of the data from all nodes.

[0180] For example, if there are four nodes participating in distributed training: node 0 contains data A, node 1 contains data B, node 2 contains data C, and node 3 contains data D, after Allreduce communication, each node has data X, where X = A + B + C + D. After Allgather communication, each node contains data A through D. After ReduceScatter communication, data A is divided into A0 through A3, data B into B0 through B3, data C into C0 through C3, and data D into D0 through D3. After communication, node 0 contains data A0 + B0 + C0 + D0, node 1 contains data A1 + B1 + C1 + D1, node 2 contains A2 + B2 + C2 + D2, and node 3 contains A3 + B3 + C3 + D3. It should be understood that the above examples are for illustrative purposes only. Other types of ensemble communication are also included, but will not be listed here.

[0181] To facilitate a better understanding of this application, the following will be combined with... Figure 8 and Figure 9 Taking a distributed training scenario for AI models, specifically a ring traffic pattern-based aggregate communication scenario, as an example, this paper illustrates the data communication method provided in this application. Figure 8 Let's take Allgather, a collection communication method, as an example to illustrate this. Figure 9 This example uses allreduce as an example of a set communication algorithm. Other set communication algorithms such as ReduceScatter can also be referenced. Figure 8 and Figure 9 The relevant descriptions will not be repeated here.

[0182] Figure 8 This is a flowchart illustrating the steps of the data communication method provided in this application in an application scenario. Figure 8 In the scenario shown, the AI ​​model uses a tensor parallel strategy, the set communication method is the neighbor exchange algorithm in the AllGatherv algorithm, and the flow pattern is a ring pattern, where adjacent nodes need to communicate with each other.

[0183] like Figure 8 As shown, the communication system includes nodes 0 to 3. The data communication method of this application used in this scenario may include the following steps:

[0184] S710: Nodes 0 through 3 each register their own memory domains.

[0185] In the specific implementation, the operation steps are the same on each node. Taking node 0 as an example, node 0 can execute S510A~S540A as described above. The communication library on the node pre-registers a memory domain of the first capacity to the RDMA hardware, obtains the key and starting address of the memory domain, and then divides the memory domain into multiple blocks, recording the address information of each block (including the starting address and offset). When the application training the AI ​​model requests to register a memory domain with the communication library, the communication library can allocate the multiple blocks required by the application and obtain the application's exchange information. Figure 8 In the example, each node is allocated 4 blocks. The exchanged information includes at least the address information of the above 4 blocks and the key of the memory domain. Of course, the exchanged information can also include the address information of each block in the entire memory domain. For details, please refer to the relevant description of S540A. This application does not make specific limitations.

[0186] Similarly, nodes 1 through 3 can each register their memory domains according to the description of node 0, and obtain their respective exchange information. This is for differentiation. Figure 8 In the example, the four blocks of node 0 are represented by A0 to A3, the four blocks of node 1 are represented by B0 to B3, the four blocks of node 2 are represented by C0 to C3, and the four blocks of node 3 are represented by D0 to D3.

[0187] S720: Nodes 0 to 3 exchange information with each other.

[0188] In practice, only nodes with communication needs exchange information with each other. Nodes without communication needs do not need to exchange information. Whether there is a communication need between nodes can be determined based on the parallel strategy used in model training. Here, the parallel strategy refers to the parallel strategy used during AI model training. Common parallel training algorithms include data parallelism, tensor parallelism, and pipelined parallelism. Data parallelism refers to dividing the dataset and distributing it to multiple nodes for training. Tensor parallelism refers to distributing the model parameters to multiple nodes for training. Pipeline parallelism refers to dividing the model according to network layers and distributing it to multiple nodes for training. Of course, there are many other types of parallel strategies, which will not be listed here. It should be noted that hybrid parallel strategies can also be used in the distributed training of AI models, such as a hybrid parallel strategy that uses both data parallelism and tensor parallelism, or other hybrid parallel strategies. This application does not make specific limitations.

[0189] Each parallel strategy may support different traffic patterns. Here, traffic pattern refers to the way data is exchanged between accelerator cards, describing how data flows between computing nodes when a distributed training system executes a parallel strategy. It should be understood that the parallel strategy focuses on how to break down, distribute, and coordinate the overall training task across various accelerator cards, while the traffic pattern focuses on how specific data is transferred between them. In specific implementations, traffic patterns may include rings, havling and doubling (HD), double binary trees (DBtree), etc., and this application does not impose specific limitations.

[0190] exist Figure 8 In the example shown, the set communication method is the neighbor exchange algorithm in the AllGatherv algorithm, and the flow pattern is ring mode, meaning adjacent nodes need to communicate with each other. Node 0 and Node 1 send data to each other, Node 1 and Node 2 send data to each other, Node 2 and Node 3 send data to each other, and Node 3 and Node 1 send data to each other. Therefore, adjacent nodes have a communication need and need to exchange data. Non-adjacent nodes do not have a communication need and do not need to exchange data. For example, Node 1 and Node 3 do not have a communication need, and Node 0 and Node 2 do not have a communication need and therefore do not need to exchange information.

[0191] In practice, the step of sending and exchanging information can occur before the collective communication begins. Specifically, it can occur before the AI ​​training task starts or during the AI ​​training task. In short, it must occur before the collective communication begins to avoid having to wait for the S720 to finish executing before RDMA can start when the collective communication begins.

[0192] Understandable. Figure 8 The example given is the distributed training of an AI model. In this scenario, the parallel strategy can be used to determine whether there is a communication need between nodes. Similar methods can be used in other scenarios to analyze whether there is a communication need between nodes. Examples will not be given here.

[0193] S730: Nodes 0 to 3 send data from their memory blocks to each other for the first time.

[0194] In practical implementation, each node can use RDMA technology to unilaterally transmit data for a single block based on the exchange information obtained from the S720. Figure 8 In the example shown, node 0 can send block-A0 data to node 1, node 1 can send block-B0 data to node 2, node 2 can send block-C0 data to node 3, and node 3 can send block-D0 data to node 0. In the other direction, node 0 can send block-A1 data to node 3, node 1 can send block-B1 data to node 0, node 2 can send block-C1 data to node 1, and node 3 can send block-D1 data to node 2.

[0195] S740: Nodes 0 to 3 send data in memory blocks to each other multiple times.

[0196] In practice, each node can use RDMA technology multiple times to complete data transmission based on the exchange information obtained from the S720, until each node obtains all the data from the other nodes, ending the allgather communication. Each RDMA operation can directly perform an RDMA write operation without requiring transmission preparation, meaning there's no need to register a MapReduce, copy data, or send handshake messages. This reduces RDMA latency in frequent point-to-point communication scenarios, improving data communication efficiency.

[0197] It should be noted that the reference Figure 7 As shown in the example, in RNDV mode, before the initial RDMA operation, the receiving node can send an RTR message to the sending node to notify it that it is ready to receive data. Then, the sending node can begin executing S730, but subsequent RDMA operations no longer require sending RTR messages; S730 can be executed directly.

[0198] Meanwhile, after the sending node writes the data to the receiving end using RDMA technology, it can directly send ATP messages without waiting for the receiving end to return ACK messages. The receiving end can use the network card with bus ordering characteristics to continuously return ACK messages in the order of reception, so that the sending end does not need to wait for ACK messages before returning ATP messages. Each RDMA can end early, and multiple RDMAs can save a lot of communication waiting time.

[0199] Simply put, in Figure 8 In the example, if the RNDV mode is used, after each node sends the data in the memory block, it can directly send an ATP message. After each node receives the data, it can continuously return ACK messages in the order of reception. If there is a transmission failure, the sender can re-perform the RDMA operation based on the failure message carried in the ACK message, or report the write failure message. There are no specific limitations for each application.

[0200] Understandably, in the above-mentioned allgather scenario, the technical solution of this application executes S710~S720 before the collective communication. After each node obtains the address information of the memory block of the node to be communicated, when the nodes need to perform collective communication, they can directly perform RDMA based on the address information of the memory block. Each RDMA does not require transmission preparation operations, does not require MR registration, does not require sending handshake messages, and does not require copying data. The more RDMA times, the more overhead is saved and the higher the communication efficiency is improved.

[0201] Figure 9 This is a flowchart illustrating the steps of the data communication method provided in this application in another application scenario. Figure 9 In the scenario shown, the AI ​​model uses a tensor parallel strategy, the allreduce algorithm for set communication, and a ring mode for traffic flow. Adjacent nodes require unidirectional communication. That is, node 0 sends data to node 1, but node 1 does not need to send data back to node 0; this can be termed unidirectional communication. However... Figure 9 In the scenario shown, node 0 and node 1 need to send data to each other, which can be called bidirectional communication.

[0202] like Figure 9 As shown, the communication system includes nodes 0 to 3. The data communication method of this application used in this scenario may include the following steps:

[0203] S810: Nodes 0 through 3 each register their own memory domains.

[0204] This step is the same as described in S710. After each node registers a memory domain, it obtains the memory domain's key and starting address. Then, it divides the memory domain into multiple blocks and records the address information of each block (including the starting address and offset). When an application training an AI model requests to register a memory domain with the communication library, the communication library can allocate the multiple blocks required by the application and obtain the application's exchange information. Figure 9 In the example, each node is allocated 4 blocks, which is the same as... Figure 8 The descriptions are the same, so I will not repeat them here.

[0205] Therefore, after each node registers its own memory domain, divides the blocks, and allocates blocks to the application, the exchange information obtained by each node includes at least the address information of the above four blocks, as well as the key of the memory domain. Of course, the exchange information may also include the address information of each block in the entire memory domain. For details, please refer to the relevant descriptions of S540A and S710. This application does not make any specific limitations.

[0206] S820: Nodes 0 to 3 send and exchange information.

[0207] In practice, only nodes with communication needs exchange information; nodes without communication needs do not need to exchange information. Furthermore, in Figure 9 In the example shown, the set communication method is the allreduce algorithm, the traffic mode is ring mode, each node sends data to the next node, and the transmission direction is unidirectional, that is, the next node will not send data to the current node. Therefore, when sending exchange information, the current node only needs to obtain the exchange information from the next node, and does not need to send its own exchange information to the next node, because the next node will not send data to the current node.

[0208] It should be noted that the current node is the node that needs to send data, and the next node of the current node refers to the node that needs to receive the data sent by the current node, for example... Figure 9 In the example, when node 0 sends data to node 1, and node 0 is the current node, node 1 is the next node of node 0. Node 1 will not send data to node 0, so node 1 only needs to send exchange information to node 0. Similarly, when node 1 is the current node, and node 2 is the next node of node 1, when node 1 sends data to node 2, node 2 will not send data to node 1, so node 2 only needs to send exchange information to node 1. And so on, the direction of information exchange can be as follows. Figure 9 As shown, we will not go into detail here.

[0209] It should be noted that the implementation details of S820 can be found in the relevant description of S730, and will not be repeated here.

[0210] S830: Nodes 0 to 3 send data from the memory block for the first time.

[0211] In practical implementation, each node can use RDMA technology to unilaterally transmit data for a single block based on the exchange information obtained from the next node via S820. Figure 9 In the example shown, node 0 sends block-A0 data to node 1, node 1 sends block-B1 data to node 2, node 2 sends block-C2 data to node 3, and node 3 sends block-D3 data to node 0. After each node receives the data, it adds the received data to its local data. This is part of the allreduce algorithm.

[0212] S840: Nodes 0 to 3 send data in memory blocks to each other multiple times.

[0213] In practical implementation, each node can use RDMA technology multiple times to complete data transmission based on the exchange information obtained from the next node via S820, until the aggregated communication ends. The communication results obtained by each node can be as follows: Figure 8 As shown, details will not be elaborated here. Each RDMA operation can directly perform an RDMA write operation without requiring transmission preparation. This means there's no need to register a MapReduce, copy data, or send handshake messages, thus reducing RDMA latency in frequent point-to-point communication scenarios and improving data communication efficiency.

[0214] It should be noted that, in Figure 9 In the example, if the RNDV mode is used, after each node sends the data in the memory block, it can directly send an ATP message. After each node receives the data, it can continuously return ACK messages in the order of reception. If there is a transmission failure, the sender can re-perform the RDMA operation based on the failure message carried in the ACK message, or report the write failure message. There are no specific limitations for each application.

[0215] It should be noted that the implementation details of S830 to S840 can be found in the descriptions of S730 to S740, and will not be repeated here.

[0216] Understandably, in the above-mentioned allreduce scenario, the technical solution of this application executes S810 to S820 before the aggregate communication. After each node obtains the address information of the memory block of the node to be communicated, when the nodes need to perform aggregate communication, they can directly perform RDMA based on the address information of the memory block. Each RDMA does not require transmission preparation operations, does not require MR registration, does not require sending handshake messages, and does not require copying data. The more RDMA times there are, the more overhead is saved and the higher the communication efficiency is improved.

[0217] It should be noted that the above description of RNDV mode is based on the put transmission mode, which is the way the sender writes data to the receiver. In fact, RNDV also has a get transmission mode, in which the receiver reads data from the sender. The data transmission direction in this mode is the opposite of the put transmission mode described above. However, the technical solution of this application can also eliminate the transmission preparation operation. After the initial RDMA, each RDMA does not need to register MR or send handshake messages, which reduces the RDMA latency in frequent point-to-point communication scenarios and improves the efficiency of data communication.

[0218] Understandably, after data transmission is completed, the communication libraries of the sending and receiving ends can release memory blocks. When a new application sends a memory domain registration request to the communication library, the communication library can allocate the released memory blocks to the new memory without needing to unregister MR, thus reducing the number of times MR can be unregistered and further reducing system resource consumption.

[0219] It should be noted that the above explanation uses the example of data being written from the first node to the second node via RDMA. In the actual implementation, the data can also be read from the second node via RDMA from the first node. The data direction is reversed, but the operation process is the same. This will not be elaborated on here.

[0220] In summary, this application provides a data communication method in which a first node pre-registers a first memory domain with RDMA hardware, enabling the RDMA hardware to directly access the first memory domain via RDMA technology. The first memory domain is then divided into multiple blocks, and the address information of each block is recorded. When the first node and the second node have communication needs, they can exchange the address information of their respective memory domain blocks before communication. This allows each communication to directly perform RDMA read and write operations based on the address information of the blocks in the second node's memory domain. Each communication does not require transmission preparation operations, i.e., no data copying, no memory domain registration, and no handshake message sending. This significantly improves communication efficiency in frequent point-to-point communication scenarios.

[0221] The data communication method and data communication system provided in this application have been described in detail above. The following section will combine... Figure 10 The software structure of the computing node in the communication system provided in this application will be explained and described. The computing node can be... Figures 1-9 The first node 100 or the second node 200 shown.

[0222] Figure 10 This is a schematic diagram of the software structure of a computing node provided in this application, such as... Figure 10 As shown, the computing node 900 includes a registration unit 910, a partitioning unit 920, a switching unit 930, a data communication unit 940, a message sending unit 950, and a message receiving unit 960. It should be understood that... Figure 10 This is an exemplary partitioning method. The computing node 900 can be divided into more or fewer unit modules, and this application does not make any specific limitations.

[0223] The registration unit 910, partitioning unit 920, switching unit 930, data communication unit 940, message sending unit 950, and message receiving unit 960 can all be implemented in software or in hardware. For example, the implementation of the registration unit 910 will be described below. Similarly, the implementation of the partitioning unit 920, switching unit 930, data communication unit 940, message sending unit 950, and message receiving unit 960 can refer to the implementation of the registration unit 910.

[0224] Registration unit 910, as an example of a software functional unit, may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, registration unit 910 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0225] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0226] As an example of a hardware functional unit, the registration unit 910 may include at least one computing device, such as a server. Alternatively, the registration unit 910 may be implemented using a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system-on-a-chip (SoC), an offload card, an inference card, or any combination thereof.

[0227] The multiple computing devices included in the registration unit 910 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the registration unit 910 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the registration unit 910 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offload cards, and inference cards.

[0228] It should be noted that, in other embodiments, the steps implemented by the registration unit 910, partitioning unit 920, exchange unit 930, data communication unit 940, message sending unit 950, and message receiving unit 960 can be specified as needed. The registration unit 910, partitioning unit 920, exchange unit 930, data communication unit 940, message sending unit 950, and message receiving unit 960 respectively implement different steps in the data communication method to achieve all the functions of the first node or the second node.

[0229] The functions of the registration unit 910, the partitioning unit 920, the exchange unit 930, the data communication unit 940, the message sending unit 950, and the message receiving unit 960 are explained below.

[0230] The switching unit 930 is used to acquire the switching information of the second node during the information switching phase. This switching information includes the address information of memory blocks. The memory blocks are obtained by partitioning the memory domain of the second node. The memory domain of the second node is used by the Remote Direct Memory Access (RDMA) hardware, specifically for implementing… Figure 6 S550A to S550B in the embodiment, Figure 8 S720 in Figure 9 The S820 and its optional steps.

[0231] Data communication unit 940 is used to perform multiple communications with the second node through the RDMA hardware of the computing node during the communication phase. Each communication uses exchange information to read or write data from a memory block. Specifically, it is used to implement... Figure 6 S560 in the embodiment Figure 7 S561 to S568 in the embodiment Figure 8 S730 in Figure 9 The S830 and its optional steps.

[0232] In one possible implementation, the registration unit 910 is used to register a first memory domain of a first capacity to the RDMA hardware. The first memory domain is used for RDMA hardware to perform RDMA, and the first capacity is not less than the data size of a single RDMA communication. Specifically, it is used to implement... Figure 6 S510A to S510B in the embodiment, Figure 8 S710 in Figure 9 S810 and its optional steps.

[0233] The partitioning unit 920 is used to divide the first memory domain into multiple memory blocks and obtain the address information of each memory block, specifically for implementing... Figure 6 S520A to S520B in the embodiment, Figure 8 S710 in Figure 9 S810 and its optional steps.

[0234] The switching unit 930 is used to generate switching information for the first node based on the address information of each memory block. This switching information is used by the third node to communicate multiple times with the first node via RDMA hardware. The third node may be the same as or different from the first node. Specifically, it is used to implement... Figure 6 S530A to S540B in the embodiment, Figure 8 S710 in Figure 9 S810 and its optional steps.

[0235] In one possible implementation, the first node includes a first application and a first communication library, the second node includes a second application, and there is a communication requirement between the first application and the second application. The registration unit 910 is used to send a request to register a memory domain to the first communication library through the first application. The registration unit 910 is used to allocate at least one memory block from multiple memory blocks for the first application to use based on the request to register the memory domain through the first communication library. The registration unit 910 is used to generate exchange information of the first node based on the address information of at least one memory block through the first communication library. The exchange information of the first node includes at least the address information of at least one memory block of the first application.

[0236] In one possible implementation, the exchanged information of the second node also includes a key to the second node's memory domain. This key is obtained when the second node registers its memory domain and is used by other nodes besides the second node to remotely access the second node's memory domain via RDMA technology.

[0237] In one possible implementation, the RDMA hardware includes a bus ordering function for recording the data reception order, a message sending unit 950 for sending a Transmission Acknowledgment (ATP) message to the second node after each data communication is completed, the ATP message for notifying the receiving end that the data transmission operation has ended, and a message receiving unit 960 for receiving multiple reception completion (ACK) messages sent by the second node, wherein the ACK message is used to notify the sending end that data reception is complete, and the multiple ACK messages are sent by the second node according to the data communication order based on the bus ordering function.

[0238] In one possible implementation, the address information of the memory block includes the starting address of the memory domain to which the memory block belongs, and the offset of the memory block based on the starting address.

[0239] In one possible implementation, the communication system is used to implement ensemble communication tasks in the distributed training of artificial intelligence (AI) models. These ensemble communication tasks include one or more of the following: reducescatter, allreduce, allgather, and all-to-all.

[0240] It should be understood that the above descriptions all use computing node 900 as the first node (sender) as an example to illustrate its possible internal unit modules. When computing node 900 is the second node (receiver), its internal unit modules are the same, except that the data communication unit does not work. This is because the receiver's CPU is unaware of the RDMA process, and the message sending unit sends ACK messages, while the message receiving unit receives ATP messages, which correspond exactly to the messages sent and received by the first node. However, if the second node acts as the sender, its data communication unit will start working. Therefore, in fact, the unit modules within each computing node in the communication system can be configured according to... Figure 10 The example demonstrates the division of unit modules, and multiple computing nodes 900 can form the communication system 2000 described above.

[0241] In summary, this application provides a computing node applied in a communication system comprising multiple computing nodes. Assuming the first node is a transmitting node and the second node is a receiving node, the first node pre-registers a first memory domain with RDMA hardware, allowing the RDMA hardware to directly access the first memory domain via RDMA technology. The first memory domain is then divided into multiple blocks, and the address information of each block is recorded. When the first node and the second node have communication needs, they can exchange the address information of their respective memory domain blocks before communication. This allows each communication to directly perform RDMA read / write operations based on the address information of the blocks in the second node's memory domain, eliminating the need for transmission preparation operations (i.e., copying data, registering memory domains, and sending handshake messages). This significantly improves communication efficiency in frequent point-to-point communication scenarios.

[0242] The data communication method, communication system, and computing node provided in this application have been described in detail above. The following section will combine... Figures 11-12 The computing device provided in this application will be explained.

[0243] Figure 11 This is a schematic diagram of the structure of a computing device provided in this application, such as... Figure 11 As shown, computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, memory 1106, and communication interface 1108 communicate with each other via the bus 1102. Computing device 1100 can be a server. It should be understood that this application does not limit the number of processors, memory, and accelerator cards in computing device 1100. Computing device 1100 can be used to deploy the aforementioned... Figures 1-10 In the first node 100 of this embodiment, memory 1106 is used to store instructions, and processor 1104 is used to execute the instructions stored in memory 1106. Memory 1106 stores program code, and processor 1104 can call the program code stored in memory 1106 to perform the following operations:

[0244] During the information exchange phase, the first node obtains the exchange information of the second node. The exchange information includes the address information of memory blocks. The memory blocks are obtained by partitioning the memory domain of the second node. The memory domain of the second node is used for remote direct memory access (RDMA) hardware.

[0245] During the communication phase, the first node communicates with the second node multiple times through the first node's RDMA hardware. In each communication, the exchange information is used to read data from or write data to the memory block.

[0246] It should be understood that the memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the above. Figures 6-9 The embodiments describe the operational steps of the data communication method. The program code may include the code of one or more software units, which may be... Figure 11 The diagram shows a registration unit, a partitioning unit, an exchange unit, a data communication unit, a message sending unit, and a message receiving unit. The registration unit is used to implement… Figure 6 S510A to S510B in the embodiment, Figure 8 S710 in Figure 9 The S810 and its optional steps are described. The partitioning unit is used to implement... Figure 6 S520A to S520B in the embodiment, Figure 8 S710 in Figure 9 The S810 and its optional steps are described. The switching unit is used to implement... Figure 6 S530A to S540B in the embodiment, Figure 8 S710 in Figure 9 The S810 and its optional steps. The data communication unit is used to implement... Figure 6 S560 in the embodiment Figure 7 S561 to S568 in the embodiment Figure 8 S730 in Figure 9 The S830 and its optional steps are described. The message sending unit is used to implement... Figure 7 In the embodiment, S566 and its optional steps are implemented by the message receiving unit. Figure 7 The steps S567 and its optional steps in the embodiments will not be repeated here.

[0247] Bus 1102 can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The Unified Bus is also known as the Lingqu Bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 11The bus 1102 is represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1102 may include a path for transmitting information between various components of the computing device 1100 (e.g., memory 1106, processor 1104, communication interface 1108). The unified bus may also be referred to as the Lingqu bus.

[0248] The processor 1104 may include any one or more of the following computing devices: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP) or digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc.

[0249] Memory 1106 may include volatile memory, such as random access memory (RAM). Memory 1106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 1106 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0250] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 1106, or two or more types of storage media can be configured to realize the function of memory 1106. This application does not limit this.

[0251] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.

[0252] Furthermore, the accelerator card 1107 can be implemented through computing devices such as DPU, NPU, GPU, XPU, SoC, or computing units such as AI chips. The accelerator card 1107 is used to implement distributed training of AI models. The accelerator card 907 includes a processing unit and a power supply circuit. The power supply circuit is used to supply power to the processing unit, enabling the processing unit to perform various tasks in distributed training.

[0253] It should be understood that the accelerator card 1107 can be understood as the processor used to train the AI ​​model mentioned above, while the processor 1104 refers to the processing core in the computing device 1100, such as the CPU. The two are different concepts.

[0254] It should be noted that the aforementioned computing node 900 can also be the second node mentioned above. The unit modules in the memory 1106 can be executed by the processor 1104 to implement the operation steps that the second node needs to perform in the data communication method described above. This will not be elaborated on here.

[0255] This application also provides a computing device cluster for deploying the aforementioned communication system 2000, such as... Figure 12 As shown, Figure 12 This is an example diagram of a computing device cluster provided in this application, which includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0256] In a specific implementation, the same implementation can be stored in the memory 1106 of one or more computing devices 1100 in the computing device cluster. Figures 6 to 9 The instructions describe the operational steps of the data communication method. Alternatively, different implementations may be stored separately in the memory 1106 of one or more computing devices 1100. Figures 6 to 9 The instructions for the operational steps of the described data communication method, in other words, a combination of one or more computing devices 1100 can jointly execute the instructions for the operational steps required to implement the data communication method.

[0257] This application also provides a chip that may include a processing unit and a power supply circuit. The power supply circuit can supply power to the processing unit, enabling the power supply unit to perform the operation steps of the first node or the second node in the above-described data communication method.

[0258] This application also provides a chip system comprising multiple chips as described above, which is used to deploy the communication system described above and implement the data communication method described above.

[0259] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the aforementioned data communication method.

[0260] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the aforementioned data communication method.

[0261] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A data communication method, characterized in that, The method is applied to a communication system, the communication system including a first node and a second node, and the method includes: During the information exchange phase, the first node obtains the exchange information of the second node. The exchange information includes the address information of memory blocks. The memory blocks are obtained by partitioning the memory domain of the second node. The memory domain of the second node is used for remote direct memory access (RDMA) hardware. During the communication phase, the first node communicates with the second node multiple times through the first node's RDMA hardware. In each communication, the exchange information is used to read data from or write data to the memory block.

2. The method according to claim 1, characterized in that, The method further includes: The first node registers a first memory domain of a first capacity to the first node's RDMA hardware. The first memory domain is used for the first node's RDMA hardware to perform RDMA communication. The first capacity is not less than the amount of data the first node performs in a single RDMA communication. The first node divides the first memory domain into multiple first memory blocks and obtains the address information of each first memory block; The first node generates the exchange information of the first node based on the address information of each first memory block; The first node sends its exchange information to the third node. The exchange information is used by the third node to communicate with the first node multiple times through the third node's RDMA hardware.

3. The method according to claim 2, characterized in that, The first node includes a first application and a first communication library, and the second node includes a second application. There is a communication requirement between the first application and the second application. The first node generates exchange information based on the address information of each memory block, including: The first application of the first node sends a request to the first communication library to register a memory domain; Based on the request from the registered memory domain, the first communication library of the first node allocates at least one first memory block from the plurality of first memory blocks for use by the first application; The first communication library of the first node generates the exchange information of the first node based on the address information of the at least one first memory block. The exchange information of the first node includes at least the address information of the at least one first memory block of the first application.

4. The method according to any one of claims 1 to 3, characterized in that, The exchange information of the second node also includes the key of the memory domain of the second node, which is obtained when the second node registers the memory domain of the second node. The key is used by other nodes other than the second node to access the memory domain of the second node through RDMA communication.

5. The method according to any one of claims 1 to 4, characterized in that, The RDMA hardware includes a bus ordering function, which is used to record the data reception order. The method further includes: Each time the first node completes a data communication, it sends a Transmission Confirmation (ATP) message to the second node. The ATP message is used to notify the second node that the data communication operation has ended. The first node receives multiple ACK messages sent by the second node, wherein the ACK messages are used to notify the first node that data reception or data writing is complete, and the multiple ACK messages are sent by the second node in the order of data communication based on the bus ordering function.

6. The method according to any one of claims 1 to 5, characterized in that, The address information of the memory block includes the starting address of the memory domain to which the memory block belongs, and the offset of the memory block based on the starting address.

7. The method according to any one of claims 1 to 6, characterized in that, The communication system is used to implement the ensemble communication task in the distributed training of artificial intelligence (AI) models. The ensemble communication task includes one or more of the following: reducescatter, allreduce, allgather, and all-to-all.

8. A computing node, characterized in that, The computing nodes include: The switching unit is used to obtain the switching information of the second node during the information switching phase. The switching information includes the address information of the memory block. The memory block is obtained by dividing the memory domain of the second node. The memory domain of the second node is used for remote direct memory access (RDMA) hardware. The data communication unit is used to communicate with the second node multiple times during the communication phase through the RDMA hardware of the computing node, and to read or write data from the memory block using the exchange information in each communication.

9. The computing node according to claim 8, characterized in that, The computing node also includes: A registration unit is used to register a first memory domain of a first capacity to the RDMA hardware of the computing node. The first memory domain is used for the RDMA hardware of the first node to perform RDMA communication. The first capacity is not less than the amount of data that the first node performs in a single RDMA communication. A partitioning unit is used to divide the first memory domain into multiple memory blocks and obtain the address information of each memory block; The switching unit is used to generate switching information for the computing node based on the address information of each memory block. The switching information of the computing node is used for the third node to communicate with the computing node multiple times through RDMA hardware.

10. The computing node according to claim 9, characterized in that, The computing node includes a first application and a first communication library, and the second node includes a second application. There is a communication requirement between the first application and the second application. The registration unit is used to send a request to register a memory domain to the first communication library through the first application; The registration unit is configured to allocate at least one memory block from the plurality of memory blocks to the first application based on a request from the registered memory domain via the first communication library; The registration unit is configured to generate exchange information of the computing node based on the address information of the at least one memory block through the first communication library, wherein the exchange information of the computing node includes at least the address information of at least one memory block of the first application.

11. The node according to any one of claims 8 to 10, characterized in that, The exchange information of the second node also includes the key of the memory domain of the second node. The key is obtained when the second node registers the memory domain of the second node. The key is used to allow other nodes besides the second node to remotely access the memory domain of the second node through RDMA technology.

12. The node according to any one of claims 8 to 11, characterized in that, The RDMA hardware includes a bus ordering function for recording the data reception order. The node also includes a message sending unit and a message receiving unit. The message sending unit is used to send a transmission confirmation (ATP) message to the second node after each data communication is completed. The ATP message is used to notify the receiving end that the data transmission operation has ended. The message receiving unit is used to receive multiple ACK messages sent by the second node, wherein the ACK message is used to notify the sending end that data reception is complete, and the multiple ACK messages are sent by the second node in the order of data communication based on the bus ordering function.

13. The node according to any one of claims 8 to 12, characterized in that, The address information of the memory block includes the starting address of the memory domain to which the memory block belongs, and the offset of the memory block based on the starting address.

14. The node according to any one of claims 8 to 13, characterized in that, The communication system is used to implement the ensemble communication task in the distributed training of artificial intelligence (AI) models. The ensemble communication task includes one or more of the following: reducescatter, allreduce, allgather, and all-to-all.

15. A distributed system, characterized in that, The distributed system includes multiple computing nodes, each computing node implementing the operation steps of the method as described in any one of claims 1 to 7.

16. A computing device, characterized in that, The computing device includes a processor and a memory, the memory being used to store code, the code being executed by the processor, such that the computing device performs the operational steps of the method as described in any one of claims 1 to 7.

17. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, cause the computing device to perform the operational steps of the method as described in any one of claims 1 to 7.

18. A computer program product containing instructions, characterized in that, When the instructions are executed by the computing device, the computing device performs the operational steps of the method as described in any one of claims 1 to 7.

Images