Communication system, communication method and related apparatus

Abstract

The present application provides a communication system, a communication method and a related apparatus. When a member participating in collective communication is to use remote data of other members and local data of the present member to carry out a calculation, a transmission unit not only acquires the remote data for the present member, but also reads, for the present member, the local data according to a storage address described by a reception request of the present member, and the transmission unit carries out the calculation on the remote data and the local data, thereby helping to reduce the number of memory operations involved in the collective communication process and reduce the number of interactions between different transmission units, such that transmission resources are saved, the time delay of executing the calculation is also reduced, and the execution efficiency of collective communication is improved.

Description

Communication system, communication method and related devices

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 22, 2024, with application number CN202410350153.2 and invention name “Communication system, communication method and related devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computers, and more specifically, to a communication system, a communication method, and related devices. Background Art

[0003] With the rapid development of high-performance computing (HPC) clusters and artificial intelligence (AI), increasing amounts of data require more processes to collaborate and process in parallel. In scenarios where multiple processes collaborate, collective communication is necessary. In collective communication, topologies involving reduced computation (MPI_reduce) or reduced and broadcast computation (MPI_allreduce) generally account for a large proportion. Calculation is a crucial component of MPI_reduce or MPI_allreduce, and the efficiency of calculation significantly impacts the efficiency of collective communication.

[0004] All or part of the data a node needs to perform computational operations typically comes from multiple other nodes. Nodes typically transmit data to other nodes through remote direct memory access (RDMA). The RDMA module writes the received data to memory, from which the node then reads the data and performs computational operations on it. This collective communication process involves numerous memory operations and interactions between RDMA modules. This not only wastes significant transmission resources but also increases computational latency, hindering efficiency. Summary of the Invention

[0005] The present application provides a communication system, a communication method and related devices, which help reduce the number of memory operations involved in the collective communication process and the number of interactive operations between RDMA modules, which not only saves transmission resources, but also helps to reduce the delay in executing computing operations, which is beneficial to improving the execution efficiency of computing operations and improving the execution efficiency of collective communications.

[0006] In a first aspect, the present application provides a communication system, which may include multiple nodes, two different nodes of which are respectively referred to as a first node and a second node. The second node may include a transmission unit and one or more processing units, and the one or more processing units include a first processing unit. The first node is used to send a message to the transmission unit, wherein the message encapsulates first data transmitted to the first processing unit. The transmission unit is used to receive the message and obtain a first receive request generated by the first processing unit, wherein the first receive request includes first address information describing a first storage location. The transmission unit is also used to read the second data in the first storage location according to the first address information, and perform a calculation operation on the first data and the second data to obtain a calculation result.

[0007] The transmission unit is a unit with computing capabilities. The present application proposes that the transmission unit is not only used to perform data transmission tasks, such as receiving messages, but can also be used to perform computing operations on the received data according to the first receiving request generated by the first processing unit, rather than writing the received data to the memory of the first processing unit and then having the first processing unit read the data from the memory again for computing. By offloading the computing tasks of the first processing unit to the transmission unit, it is beneficial to avoid or reduce the memory write operations and memory read operations on the received first data in the second node, thereby reducing the number of memory operations in the system, reducing the latency and memory bandwidth consumption of computing operations, improving the execution efficiency of computing operations, and improving the execution efficiency of collective communications.

[0008] In addition, since the transmission unit is used to determine the storage location (i.e., the first storage location) of the local operand (i.e., the second data) of the computing operation from the receiving request generated by the first processing unit as the receiving end, rather than determining the first storage location from the message sent by the sending end, this is not only beneficial to avoid the message carrying the first location information of the first storage location and reducing the size of the message, but also beneficial to avoid the first processing unit notifying the first location information to the first node before the first node sends the message, reducing the interactive operations between the first processing unit and the first node, saving transmission resources between nodes, reducing the delay in executing computing operations, improving the execution efficiency of computing operations, and improving the execution efficiency of collective communications.

[0009] Optionally, before sending the message, the first node may negotiate with the second node to confirm that the data sent to the first processing unit needs to be sent to the transmission unit, and confirm the address of the transmission unit.

[0010] Optionally, the message also encapsulates first indication information, and the first indication information is used to determine the receiving request generated by the first processing unit from the receiving requests generated by the one or more processing units; the transmission unit is specifically used to obtain the first receiving request according to the first indication information in the message.

[0011] In this way, when the second node also includes other processing units in addition to the processing unit, the transmission unit is also used to obtain the receiving request generated by the other processing units. The transmission unit obtains the first receiving request according to the first indication information, which is conducive to obtaining the receiving request generated by the processing unit from the receiving requests generated by each processing unit, improving the accuracy of obtaining the first receiving request, and thus improving the accuracy of communication.

[0012] In addition, when the instruction to be executed by the first processing unit not only instructs the use of the data in the first storage location and the first data of the first node to perform the calculation operation, but also instructs the use of other data sent by other nodes and the data in the first storage location to perform other calculation operations, the transmission unit can obtain multiple receive requests generated in sequence by the first processing unit. When the exchange law is satisfied between multiple calculation operations, after receiving the first data, the transmission unit obtains the receive request generated by the first processing unit according to the first indication information, and uses the obtained receive request to perform calculation operations on the first data, which is beneficial for the transmission unit to perform calculation operations on the data that arrives first, thereby reducing the execution delay of instructions and improving the efficiency of collective communication.

[0013] Optionally, the communication system further includes other nodes in addition to the first node and the second node, and the receiving request generated by the first processing unit includes a receiving request for receiving data from the first node and a receiving request for receiving data from the other nodes, and the first indication information is further used to determine the receiving request for receiving data from the first node from the receiving request generated by the first processing unit. In this way, even if the multiple computing operations indicated by the instruction to be executed by the first processing unit do not satisfy the commutative law, after the transmission unit receives the first data transmitted by the first node to the first processing unit, it obtains the receiving request generated by the first processing unit for receiving data from the first node according to the first indication information, and uses the receiving request to perform the corresponding computing operation, which is conducive to ensuring the correct execution of the instruction and improving the accuracy of the collective communication.

[0014] Optionally, the first data is a portion of the target data transmitted by the first node to the first processing unit, and the receiving request generated by the first processing unit for receiving data from the first node includes a receiving request for receiving the first data and a receiving request for receiving other data in the target data except the first data, and the first indication information is also used to determine the receiving request for receiving the first data from the receiving request generated by the first processing unit for receiving data from the first node. In this way, even if the first node transmits data to the first processing unit through multiple messages, the first processing unit generates multiple receiving requests for receiving different data from the first node. Since the transmission unit obtains the receiving request for receiving the first data from the multiple receiving requests according to the first indication information after receiving the first data, it is beneficial to improve the accuracy of the transmission unit in processing the first data and improve the accuracy of collective communication.

[0015] Optionally, the first receiving request further includes second indication information, where the second indication information is used to indicate that the computing operation is to be performed on the second data and the first data. By using the second indication information, the present application can flexibly indicate whether to process each message sent by the first node to the transmission unit in accordance with the receiving request.

[0016] Optionally, the first receiving request further includes third indication information, where the third indication information is used to indicate the type of the computing operation. By using the third indication information in the receiving request, the present application can flexibly indicate the type of computing operation to be performed on each message sent by the first node to the transmission unit.

[0017] Optionally, the transmission unit is further configured to pre-process the first data and / or the second data before performing the computing operation on the first data and the second data.

[0018] Optionally, the first reception request further includes second address information describing a second storage location, where the second storage location is used to store the calculation result; and the transmission unit is further configured to write the calculation result to the second storage location according to the second address information. This facilitates flexible indication of the storage location of the calculation result.

[0019] Optionally, the transmission unit is further configured to write the calculation result into the first storage location according to the first address information. This helps to reduce the length of the first receiving request and save storage resources used to store the receiving request.

[0020] Optionally, the first node and the second node may perform scheduling synchronization to ensure that the first node sends the message after the first processing unit in the second node submits the first receive request. For example, after the first processing unit submits the receive request to the transmission unit, it notifies the first node that the first receive request has been submitted. Only after receiving this notification can the first node send the message to the transmission unit. This facilitates the transmission unit to successfully query the first processing unit's first receive request upon receiving the message, thereby reading the second data and performing computational operations on the first data and the second data.

[0021] Optionally, the first node and the second node may not perform scheduling synchronization, that is, the first node sending the message is independent of whether the first processing unit submits the first receive request, and the first node can send the message regardless of whether the first processing unit has submitted the first receive request. In this way, if the first node prepares the data before the first processing unit, the first node can send the message before the first processing unit submits the first receive request, which is beneficial for the transmission unit to receive the first data with a shorter delay after receiving the first receive request, and may even receive the first data at the same time as receiving the first receive request, thereby reducing the delay in completing the computing operation.

[0022] Optionally, the message indicating that the transmission unit has obtained the first receive request can be the message retransmitted once or multiple times by the first node. For example, before the first node sends the retransmitted message, the first node sends the message to the transmission unit, but the transmission unit cannot obtain the first receive request based on the message. The transmission unit can discard the message and instruct the first node to retransmit the message. In this way, it is beneficial to support scheduling-free synchronization between the first node and the second node, so that the transmission unit can receive the first data with a smaller delay after receiving the first receive request, and may even receive the first data at the same time as receiving the receive request, which is beneficial to reducing the delay in completing the computing operation and improving the execution efficiency of the computing operation.

[0023] Optionally, the calculation operation is a collective communication calculation. For example, the first node and the second node or the processing unit in the first node (such as the source of the first data, referred to as the second processing unit) and the processing unit in the second node (such as the first processing unit) are different members in the member group participating in the collective communication, and the calculation operation is a calculation operation in the collective communication. Since the progress of collective communication performed by different members in the member group is generally consistent, for example, the moment when the second processing unit prepares the first data and the moment when the first processing unit prepares the second data are relatively close. Therefore, even if the second processing unit submits a sending request to send the first data after preparing the first data, when the first data arrives at the transmission unit, the first processing unit is usually also ready for the second data and submits a receiving request, which is beneficial to reduce the number of times the first node retransmits the message, thereby helping to reduce the delay of collective communication and save transmission resources between the first node and the second node.

[0024] Optionally, the collective communication calculation includes collective communication protocol calculation, or collective communication protocol and broadcast calculation. Optionally, the calculation operation includes at least one of an addition operation, a maximum value operation, a sum operation, an or operation, an exclusive or operation, and a minimum value operation.

[0025] Optionally, the message is a Remote Direct Memory Access protocol RDMA message or a Transmission Control Protocol TCP message.

[0026] Optionally, based on the message being an RDMA message, the transmission unit includes an RDMA module, which may specifically be an RDMA engine or an RDMA network interface controller (RDMA network interface controller, RNIC), which may also be called an RDMA network card.

[0027] Optionally, based on the fact that the message is a TCP message, the transmission unit may include a TCP protocol stack.

[0028] Optionally, the transmission unit may include other modules in addition to the RDMA module or the TCP protocol stack. This application refers to such other modules as home agents (HAs). After the RDMA module or the TCP protocol stack of the transmission unit receives the message and queries the storage address described by the reception request, it may send a calculation command and the first data to the HA. The calculation command is used to instruct the HA to read the second data from the storage address, perform a calculation operation on the first data and the second data, and write the calculation result to the memory.

[0029] HA can communicate with the RDMA module or TCP protocol stack through the bus, and communicate with the memory through the non-bus interface, which helps to avoid the RDMA module or TCP protocol stack reading the second data from the memory through the bus, and writing the calculation results to the memory module through the bus, thereby reducing the latency and bandwidth consumption of collective communication.

[0030] In one possible design, in another implementation of the first aspect of the present application, the memory may include dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), or double data rate synchronous dynamic random access memory (DDR SDRAM), etc., without limitation.

[0031] In one possible design, in another implementation of the first aspect of the present application, the communication system may further include a memory.

[0032] In one possible design, in another implementation of the first aspect of the present application, the transmission unit is configured to receive a receive request from the first processing unit and store the receive request in a memory, for example, in a receive queue in the memory. The transmission unit is configured to query the memory or the receive queue in the memory for the receive request after receiving the first message. The transmission unit is further configured to delete the receive request after querying the receive request or receiving and processing the first data as indicated by the receive request.

[0033] Optionally, the transmission unit is further used to notify the processing unit of the completion of the calculation operation after obtaining the calculation result.

[0034] In one possible design, the transmission unit is further used to report an interrupt to the first processing unit after writing the calculation result to the memory, and the interrupt is used to indicate that the receiving request has been completed.

[0035] In one possible design, the transmission unit is further configured to receive a poll from the first processing unit to return completion of the reception request.

[0036] In a second aspect, the present application provides a communication method, which can be applied to a communication system, wherein the communication system may include a first node and a second node, the second node may include a transmission unit and one or more processing units, and the one or more processing units include a first processing unit. The method may include: the first node sends a message to the transmission unit, wherein the message encapsulates first data to be transmitted to the first processing unit. The transmission unit receives the message and obtains a first receive request generated by the first processing unit, wherein the first receive request includes first address information describing a first storage location. The transmission unit reads the second data in the first storage location according to the first address information, and performs a calculation operation on the first data and the second data to obtain a calculation result.

[0037] Optionally, before sending the message, the first node may negotiate with the second node to confirm that the data sent to the first processing unit needs to be sent to the transmission unit, and confirm the address of the transmission unit.

[0038] Optionally, the message also encapsulates first indication information, and the first indication information is used to determine the receiving request generated by the first processing unit from the receiving requests generated by the one or more processing units; the transmission unit obtains the first receiving request according to the first indication information in the message.

[0039] Optionally, the communication system further includes other nodes in addition to the first node and the second node, and the receiving request generated by the first processing unit includes a receiving request for receiving data from the first node and a receiving request for receiving data from the other nodes, and the first indication information is further used to determine the receiving request for receiving data from the first node from the receiving request generated by the first processing unit. In this way, even if the multiple computing operations indicated by the instruction to be executed by the first processing unit do not satisfy the commutative law, after the transmission unit receives the first data transmitted by the first node to the first processing unit, it obtains the receiving request generated by the first processing unit for receiving data from the first node according to the first indication information, and uses the receiving request to perform the corresponding computing operation, which is conducive to ensuring the correct execution of the instruction and improving the accuracy of the collective communication.

[0040] Optionally, the first data is a portion of the target data transmitted by the first node to the first processing unit, and the receiving request generated by the first processing unit for receiving data from the first node includes a receiving request for receiving the first data and a receiving request for receiving other data in the target data except the first data, and the first indication information is also used to determine the receiving request for receiving the first data from the receiving request generated by the first processing unit for receiving data from the first node. In this way, even if the first node transmits data to the first processing unit through multiple messages, the first processing unit generates multiple receiving requests for receiving different data from the first node. Since the transmission unit obtains the receiving request for receiving the first data from the multiple receiving requests according to the first indication information after receiving the first data, it is beneficial to improve the accuracy of the transmission unit in processing the first data and improve the accuracy of collective communication.

[0041] Optionally, the first receiving request further includes second indication information, where the second indication information is used to indicate that the computing operation is to be performed on the second data and the first data. By using the second indication information, the present application can flexibly indicate whether to process each message sent by the first node to the transmission unit in accordance with the receiving request.

[0042] Optionally, the first receiving request further includes third indication information, where the third indication information is used to indicate the type of the computing operation. By using the third indication information in the receiving request, the present application can flexibly indicate the type of computing operation to be performed on each message sent by the first node to the transmission unit.

[0043] Optionally, the transmission unit preprocesses the first data and / or the second data before performing the computing operation on the first data and the second data.

[0044] Optionally, the first reception request further includes second address information describing a second storage location, where the second storage location is used to store the calculation result; and the transmission unit writes the calculation result to the second storage location according to the second address information. This facilitates flexible indication of the storage location of the calculation result.

[0045] Optionally, the transmission unit writes the calculation result into the first storage location according to the first address information. This helps to reduce the length of the first receive request and save storage resources used to store the receive request.

[0046] Optionally, the first node and the second node may perform scheduling synchronization to ensure that the first node sends the message after the first processing unit in the second node submits the first receive request. For example, after the first processing unit submits the receive request to the transmission unit, it notifies the first node that the first receive request has been submitted. Only after receiving this notification can the first node send the message to the transmission unit. This facilitates the transmission unit to successfully query the first processing unit's first receive request upon receiving the message, thereby reading the second data and performing computational operations on the first data and the second data.

[0047] Optionally, the first node and the second node may not perform scheduling synchronization, that is, the first node sending the message is independent of whether the first processing unit submits the first receive request, and the first node can send the message regardless of whether the first processing unit has submitted the first receive request. In this way, if the first node prepares the data before the first processing unit, the first node can send the message before the first processing unit submits the first receive request, which is beneficial for the transmission unit to receive the first data with a shorter delay after receiving the first receive request, and may even receive the first data at the same time as receiving the first receive request, thereby reducing the delay in completing the computing operation.

[0048] Optionally, the message indicating that the transmission unit has obtained the first receive request can be the message retransmitted once or multiple times by the first node. For example, before the first node sends the retransmitted message, the first node sends the message to the transmission unit, but the transmission unit cannot obtain the first receive request based on the message. The transmission unit can discard the message and instruct the first node to retransmit the message. In this way, it is beneficial to support scheduling-free synchronization between the first node and the second node, so that the transmission unit can receive the first data with a smaller delay after receiving the first receive request, and may even receive the first data at the same time as receiving the receive request, which is beneficial to reducing the delay in completing the computing operation and improving the execution efficiency of the computing operation.

[0049] Optionally, the calculation operation is a collective communication calculation. For example, the first node and the second node or the processing unit in the first node (such as the source of the first data, referred to as the second processing unit) and the processing unit in the second node (such as the first processing unit) are different members in the member group participating in the collective communication, and the calculation operation is a calculation operation in the collective communication. Since the progress of collective communication performed by different members in the member group is generally consistent, for example, the moment when the second processing unit prepares the first data and the moment when the first processing unit prepares the second data are relatively close. Therefore, even if the second processing unit submits a sending request to send the first data after preparing the first data, when the first data arrives at the transmission unit, the first processing unit is usually also ready for the second data and submits a receiving request, which is beneficial to reduce the number of times the first node retransmits the message, thereby helping to reduce the delay of collective communication and save transmission resources between the first node and the second node.

[0050] Optionally, the collective communication calculation includes collective communication protocol calculation, or collective communication protocol and broadcast calculation. Optionally, the calculation operation includes at least one of an addition operation, a maximum value operation, a sum operation, an or operation, an exclusive or operation, and a minimum value operation.

[0051] Optionally, the message is a Remote Direct Memory Access protocol RDMA message or a Transmission Control Protocol TCP message.

[0052] Optionally, based on the message being an RDMA message, the transmission unit includes an RDMA module, which may specifically be an RDMA engine or an RDMA network interface controller (RDMA network interface controller, RNIC), which may also be called an RDMA network card.

[0053] Optionally, based on the fact that the message is a TCP message, the transmission unit may include a TCP protocol stack.

[0054] Optionally, the transmission unit may include other modules in addition to the RDMA module or the TCP protocol stack. This application refers to such other modules as home agents (HAs). After the RDMA module or the TCP protocol stack of the transmission unit receives the message and queries the storage address described by the reception request, it may send a calculation command and the first data to the HA. The calculation command is used to instruct the HA to read the second data from the storage address, perform a calculation operation on the first data and the second data, and write the calculation result to the memory.

[0055] HA can communicate with the RDMA module or TCP protocol stack through the bus, and communicate with the memory through the non-bus interface, which helps to avoid the RDMA module or TCP protocol stack reading the second data from the memory through the bus, and writing the calculation results to the memory module through the bus, thereby reducing the latency and bandwidth consumption of collective communication.

[0056] In one possible design, in another implementation of the first aspect of the present application, the memory may include dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), or double data rate synchronous dynamic random access memory (DDR SDRAM), etc., without limitation.

[0057] In one possible design, in another implementation of the first aspect of the present application, the communication system may further include a memory.

[0058] In one possible design, in another implementation of the first aspect of the present application, the transmission unit receives a receive request from the first processing unit and stores the receive request in a memory, for example, in a receive queue in the memory. After receiving the first message, the transmission unit searches the memory or the receive queue in the memory for the receive request. The transmission unit also deletes the receive request after finding the receive request or receiving and processing the first data as instructed by the receive request.

[0059] Optionally, after obtaining the calculation result, the transmission unit notifies the processing unit of the completion of the calculation operation.

[0060] In one possible design, the transmission unit reports an interrupt to the first processing unit after writing the calculation result into the memory, where the interrupt is used to indicate that the receiving request has been completed.

[0061] In one possible design, the transmission unit also receives a poll from the first processing unit to return completion of the receive request.

[0062] The transmission unit in the second aspect can be understood with reference to the transmission unit introduced in the first aspect. The method steps and beneficial effects provided by the second aspect and possible implementation methods can be understood with reference to the corresponding actions (or functions) performed by the transmission unit in the first aspect and the corresponding implementation methods and the effects of the first aspect scheme. Other implementation methods and beneficial effects of the second aspect can be understood with reference to the other actions and effects that may be performed by the transmission unit introduced in the previous text, and will not be repeated here.

[0063] In a third aspect, the present application provides a communication method, which includes: receiving a message sent by a first node, the message encapsulating first data to be passed to a first processing unit among one or more processing units; obtaining a first receive request generated by the first processing unit, the first receive request including first address information describing a first storage location; reading second data in the first storage location according to the first address information, and performing a calculation operation on the first data and the second data to obtain a calculation result.

[0064] Optionally, the one or more processing units are deployed in a second node other than the first node, and the method is executed by the second node or by a unit in the second node (such as the transmission unit described above).

[0065] The optional implementation methods and effects of the communication method provided in the third aspect can be understood by referring to any possible implementation method and effect performed by the transmission unit in the method provided in the second aspect, and will not be repeated here.

[0066] In a fourth aspect, the present application provides a communication device, comprising: a communication module for receiving a message sent by a first node, wherein the message encapsulates first data to be passed to a first processing unit among one or more processing units; a processing module for obtaining a first receive request generated by the first processing unit, wherein the first receive request includes first address information describing a first storage location; the processing module is also used to read the second data in the first storage location according to the first address information, and perform calculation operations on the first data and the second data to obtain a calculation result.

[0067] Optionally, the one or more processing units are deployed on a second node other than the first node, and the communication device is the second node or is deployed on the second node.

[0068] The optional steps and effects performed by the communication device can be understood by referring to the optional steps and effects performed by the processing unit in the first aspect or the second aspect, and will not be repeated here.

[0069] The communication device may be a communication device, or a communication module installed in or used in conjunction with the communication device. The communication module may be a hardware module, such as a network interface controller (or network card) or a chip. Alternatively, the communication module may be a software virtual module.

[0070] The communication device may also be referred to as a host device. This application does not limit the type of the communication device. For example, the communication device may be a terminal or a server. According to the function of the communication device, the communication device may be a computing device, a storage device, or a network device.

[0071] Both the communication device and the network card can be considered as computer devices. Optionally, the computer device may include a processor and a memory, the memory being coupled to the processor, and the processor being configured to execute the method described in the second aspect or any optional embodiment of the second aspect. The software virtualization module may be generated by the processor executing the method.

[0072] Optionally, these instructions are stored in a memory external to the computer device. When these instructions are decoded and executed by the processor of the computer device, part or all of the contents of the above instructions are temporarily stored in the memory within the computer device. Optionally, part of the contents of these instructions are stored in the memory external to the computer device, and the rest of the contents of these instructions are stored in the memory within the computer device.

[0073] The chip or chip system may include one or more logic circuits to implement the method described in the second aspect or any optional embodiment of the second aspect.

[0074] In a fifth aspect, the present application provides a computer-readable storage medium storing program codes. When these program codes are executed on a computing device, the computing device executes the method described in the second aspect or any optional method of the second aspect of the present application.

[0075] A sixth aspect of the present application provides a computer program product, which, when the program code contained in the computer program product is executed by a computing device, implements the method described in the second aspect of the present application or any optional method of the second aspect.

[0076] In the present application, the first, second and third are distinguished only for the convenience of description. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is merely a way of distinguishing when describing objects with the same attributes in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] FIG1 schematically illustrates a system architecture corresponding to an existing collective communication solution;

[0078] Figures 2 to 4 schematically illustrate the system architecture corresponding to the collective communication solution of this application;

[0079] Figures 5-1 and 5-2 schematically illustrate possible processes for node S1 and node D to perform collective communication, respectively;

[0080] Figures 6 and 7 schematically illustrate possible processes for node S1, node S2, and node D to perform collective communication, respectively;

[0081] FIG8 schematically shows a possible structure of the communication device of the present application. DETAILED DESCRIPTION

[0082] The solution of the present application can be applied in a communication system, which includes multiple nodes, and different nodes are communicatively connected. The present application does not limit the scale of the communication system. For example, the communication system can be a single board, and the multiple nodes can be multiple modules on the single board. For example, the communication system is a rack server, and at least two of the multiple nodes can be different single boards in the rack server or modules deployed on different single boards. For example, the communication system is a server cluster, and at least two of the multiple nodes can be different servers in the server cluster or modules deployed on different servers. When the node is a single board or a module on a server, the module can be a physical module or a virtual module. When the module is a virtual module, the deployment location of the module can refer to the deployment location of the physical resources on which the virtual module is based. The present application does not limit the type of server. For example, the server can be a high performance computing (HPC) server, or an AI training center server, etc.

[0083] The communication system may be a system for performing collective communication, and nodes in the communication system may perform collective communication calculations. This application does not limit the mode of collective communication. The following text takes collective communication as an example of multi-point interface (MPI) reduction operation or integration. This application does not limit the function of the communication system. For example, the communication system may perform distributed training of an artificial intelligence (AI) model through collective communication.

[0084] Specifically, multiple nodes in the communication system can train the same model through a global reduction operation. During a single training of the model, different nodes can use different training data (mini-batches) to train the model and obtain the gradient of the loss function. Afterwards, the node not only sends the gradient of the loss function obtained by itself to other nodes, but also receives the gradient of the loss function from other nodes, integrates the gradient obtained by itself and the gradient received, and uses the integrated gradient to update the model. The integration includes but is not limited to at least one of an add operation, a maximum value (max) operation, an and operation, an or operation, an exclusive or (xor) operation, and a minimum value (min) operation.

[0085] Figure 1 schematically shows a possible structure of the communication system. As shown in Figure 1, the communication system may include a node S1 and a node D, and the node S1 and the node D respectively include a processing element (PE), an interconnect controller (IC), a memory controller (MC) and a dynamic random access memory (DRAM).

[0086] Among them, PE is used to train the model using training data. This application does not limit the type of PE. PE can be a hardware module, and this application does not limit the type of hardware module. For example, PE can be a graphics processing unit (GPU) or a neural processing unit (NPU) or a central processing unit (CPU). Alternatively, PE can be a software virtual module, and this application does not limit the type of software virtual module. For example, PE can be a process or application running on a node.

[0087] IC is used to transmit data for PE. For example, IC can support remote direct memory access (RDMA) protocol, and IC can transmit data for PE through RDMA message. IC can specifically be an RDMA engine, or an RDMA network interface controller (NIC), which is not limited in this application. This application does not limit the transmission protocols supported by IC. For example, IC can support other types of transmission protocols, for example, IC can support transmission control protocol (TCP), and IC can transmit data for PE through TCP message, and IC can specifically be a TCP protocol stack. This application does not limit the type of message transmitted between ICs of different nodes or the protocol used. In the following text, the message is taken as an RDMA message as an example. This application does not limit the implementation method of IC, for example, IC can be a hardware module or a software virtual module.

[0088] The MC is used to manage the DRAM and is responsible for coordinating data transmission between the PE and the memory and / or data transmission between the IC and the memory. This application does not limit the implementation of the MC. For example, the MC can be a hardware module or a software virtual module.

[0089] DRAM is used to store node data (such as model training data and the gradient of the loss function, etc.). This application does not limit the type of memory in the node. Figure 1 takes the memory in the node as DRAM as an example. Optionally, the DRAM can be replaced with other types of memory. Exemplarily, the memory in the node can be random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), flash memory, or optical memory, etc. This application does not limit the number of memories in the node, and more memories of the same or different types can be set in the node.

[0090] This application does not limit the deployment method of node S1 and node D. For example, node S1 and node D can be deployed on the same board, or on different boards in the same rack, or in different racks.

[0091] The following describes the collective communication method in the system shown in FIG1 .

[0092] After the PE in node D (denoted as PE-D) trains the model using the training data and obtains the gradient of the loss function (denoted as data 0), it can write data 0 to the DRAM in node D (denoted as DRAM-D). After the PE in node S1 (denoted as PE-S1) trains the model using the training data and obtains the gradient of the loss function (denoted as data 1), it can write data 1 to the DRAM in node S1 (denoted as DRAM-S1).

[0093] After PE-S1 writes data 1 to DRAM-S1, assuming an instruction or computation operation, such as "data N = data 0 + data 1," PE-S1 can send data 1 to the IC of node D (IC-D) via the IC of node S1 (IC-S1). After receiving data 1, IC-D can write data 1 to DRAM-D. This process can be seen in the arrow marked 1 in Figure 1.

[0094] After PE-D prepares data 0 and data 1 (i.e., writes both to DRAM-D), assuming it runs an instruction or computation operation, such as "data N = data 0 + data 1," PE-D can read data 1 from DRAM-D (refer to the process corresponding to the arrow marked 2 in Figure 1 ), and also read data 0 from DRAM-D (refer to the process corresponding to the arrow marked 3 in Figure 1 ). It then combines data 1 and data 0 to obtain data 1', and writes data 1' to DRAM-D (refer to the process corresponding to the arrow marked 4 in Figure 1 ). PE-D can then use data 1' to update model parameters.

[0095] In the collective communication process shown in Figure 1, for node D, a total of two write operations (i.e., the processes corresponding to the arrows marked 1 and 4 in Figure 1) and two read operations (i.e., the processes corresponding to the arrows marked 2 and 3 in Figure 1) are required on DRAM-D. The delay in the process of node D obtaining data 1' includes the delay of the process corresponding to the arrows marked 1 to 4 in Figure 1, as well as the calculation delay of PE-D. Among them, the process corresponding to each arrow in Figure 1 requires a read or write operation on DRAM-D.

[0096] The bandwidth between ICs of different nodes is also called interconnection bandwidth or input / output (IO) bandwidth. At present, the interconnection bandwidth has approached the DRAM bandwidth, and the two can reach the same order of magnitude. For example, David's UB interconnection bandwidth has reached 896GB / s (unidirectional), 1792GB / s (bidirectional), and the bandwidth of Custom-DRAM is 1638GB / s. If the data coming in from the UB will operate on the DRAM multiple times, then the DRAM bandwidth will become a bottleneck, which will cause delays and bandwidth consumption. Therefore, the technical problem to be solved by this application is to operate the DRAM as little as possible with the data coming in from the UB, especially to avoid multiple copies of the data, so as to reduce the delay and bandwidth consumption of the collective communication.

[0097] FIG2 schematically shows another example of a collective communication method in a system. The system shown in FIG2 can be understood by referring to the description of the system shown in FIG1 above. The difference between the collective communication process shown in FIG2 and FIG1 is that, in the collective communication process shown in FIG2, in order to reduce the number of operations on the DRAM, after the IC-D receives the RDMA message (i.e., the process corresponding to the arrow marked 1 in FIG2), it does not write the data 1 therein to the DRAM-D, but reads the data 0 from the DRAM-D (i.e., the process corresponding to the arrow marked 2 in FIG2), performs an integrated operation on the data 0 and the data 1 to obtain the data 1', and writes the operation result to the DRAM-D (i.e., the process corresponding to the arrow marked 3 in FIG2).

[0098] After IC-D writes data 1' to DRAM, PE-D no longer needs to read data 0 and data 1 separately from DRAM, nor does it need to perform integrated calculations before writing data 1' to DRAM. For node D, a total of one read operation (the process corresponding to the arrow marked 2 in Figure 2) and one write operation (the process corresponding to the arrow marked 3 in Figure 2) are required on DRAM-D. The delay in node D obtaining data 1' includes the delays corresponding to the processes marked 1 to 3 in Figure 2, as well as the computational delay of IC-D. Only the processes corresponding to arrows 2 and 3 in Figure 2 require read or write operations on DRAM-D. This helps reduce the number of memory copies, thereby reducing the latency and bandwidth consumption of collective communication.

[0099] FIG3 schematically illustrates another structure of the system. The system shown in FIG3 can be understood with reference to the system shown in FIG1 or FIG2 . However, unlike the system shown in FIG1 or FIG2 , the system shown in FIG3 also includes a home agent (HA). The HA can be a hardware module, such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., without limitation.

[0100] As the computing processing core, the PE has significant differences in frequency and communication / processing speed with the memory. Therefore, the two generally do not communicate directly, and read or write memory through their own layer 1 (L1) cache, layer 2 (L2) cache, and the public layer 3 (L3) cache. The HA, as a dedicated computing module, can directly connect to the memory through a non-bus interface (such as a private interface or a dedicated interface), and achieve matching frequency and communication / processing speed, thus enabling read or write operations to the memory without going through the cache or bus.

[0101] FIG3 also schematically illustrates another example of a collective communication method in the system. Unlike the method shown in FIG2 , in the method shown in FIG3 , after receiving the RDMA message, the IC-D does not read data 0 from the DRAM and perform an integration operation on data 0 and data 1. Instead, the task of accessing the DRAM and performing the integration operation is offloaded to the HA. For example, after receiving the RDMA message, the IC-D can send data 1 and a calculation command to the HA of this node (denoted as HA-D), which is the process corresponding to the arrow marked 1 in FIG3 . The calculation command can be used to instruct the HA to read data 0 from the DRAM and perform a calculation on data 1 and data 0. Afterwards, the HA caches data 1 according to the calculation command, reads data 0 from the DRAM (that is, the process corresponding to the arrow marked 2 in FIG3 ), performs an integration operation on data 0 and data 1 to obtain data 1', and writes the calculation result to the DRAM-D (that is, the process corresponding to the arrow marked 3 in FIG3 ).

[0102] The HA and DRAM can communicate via a non-bus interface, which can be a private interface (or dedicated interface). Therefore, when the HA accesses the DRAM, data does not need to be transmitted over the bus, which helps further save bus bandwidth. In addition, the HA can be located close to the DRAM, which helps shorten the latency of read or write operations on the DRAM, further reducing the latency of collective communication.

[0103] For ease of description, this application denotes the storage location of data 0 in DRAM-D as addr0. Before reading data 0, the IC-D or HA-D needs to determine addr0 to integrate the received data 1 with the data 0 in addr0. The following describes possible ways for the IC-D to determine addr0.

[0104] Optionally, data 1 is transmitted between node S1 and node D using a message with unilateral semantics, IC-S1 carries addr0 in the message, and IC-D determines addr0 from the message.

[0105] For example, PE-S1 can obtain addr0 in advance and, after preparing data 1, submit a transmission request to IC-S1 based on data 1 and addr0. IC-S1 then sends data 1 and addr0 to IC-D via an RDMA message. After receiving the message, IC-D can determine addr0 from the message and can then read data 0 from addr0 itself or control the HA to perform a combined operation on data 0 and the received data 1.

[0106] Neural networks are increasingly using dynamic graph frameworks. In dynamic graphs, the entire computational graph structure can be dynamically constructed and modified during program runtime. This flexibility gives dynamic graphs significant advantages in the training, debugging, and deployment of deep learning models. Taking PyTorch as an example, because its data interaction addresses are dynamic, PE-S1 cannot statically obtain addr0 in advance and must instead dynamically obtain addr0. For example, after PE-D prepares data 0, it needs to notify PE-S1 of addr0, performing address synchronization. This process can be seen in Figures 2 or 3, with the arrows marked with 0. Address synchronization increases the number of interactions between PE-D and PE-S1, wasting PE computing resources and network resources between ICs.

[0107] Figure 4 schematically illustrates another example of a collective communication method within a system. The system shown in Figure 4 can be understood by referring to the previous description of the system shown in Figure 3. However, unlike the method shown in Figure 3, in the method shown in Figure 4, data 1 is transmitted between node S1 and node D using bilateral semantic messages. That is, to transmit data 1, not only does PE-S1 submit a transmission request, or send request (SR), to IC-S1, but PE-D also submits a transmission request, or receive request (RR), to IC-D. The receive request describes addr0. This process can be seen in the arrow marked 0 in Figure 4. Accordingly, IC-D can determine addr0 from the receive request. This avoids address synchronization between PE-D and PE-S1, conserves PE computing resources and network resources between ICs, and reduces collective communication latency.

[0108] For example, after receiving a receive request, the IC-D can save the receive request in its own receive queue (RQ). The receive request saved in the RQ is also called a registered receive queue element (RQE). This process can be referred to as the process corresponding to the arrow marked 0 in Figure 4. Unlike the existing RQE that describes the write location of the data to be received (e.g., data 1) in node D, the present application proposes that the RQE can describe the storage location (e.g., addr0) of local data (e.g., data 0) in node D.

[0109] After receiving the send request, IC-S1 saves it in its send queue (SQ). The send request saved in the SQ is also called a registered send queue element (SQE). The SQE points to data 1 in DRAM-S1. IC-S1 schedules to the SQE, reads data 1 from DRAM-S1, encapsulates data 1 into an RDMA packet, and sends the RDMA packet to IC-D.

[0110] After data 1 arrives at the IC-D, the RQE is consumed. Unlike conventional RQE consumption, where the IC-D writes data 1 to the storage location described by the RQE, this application proposes that the IC-D determine addr0 in the RQE and then sends data 1 and a calculation command to the HA. The calculation command may include the address information describing addr0. This process can be seen in the process corresponding to the arrow marked 1 in Figure 4.

[0111] After receiving data 1 and the calculation command, HA can read data 0 from addr0 (that is, the process corresponding to the arrow marked 2 in Figure 4). Then, it integrates data 0 and data 1 to obtain data 1' and writes data 1' to DRAM-D (that is, the process corresponding to the arrow marked 3 in Figure 4).

[0112] As mentioned above, the node shown in Figure 4 does not need to include an HA. Accordingly, after receiving the RDMA message and obtaining the RQE, IC-D reads data 0 from addr0 described by the RQE, integrates the read data 0 and the received data 1 to obtain data 1', and writes data 1' to DRAM-D.

[0113] To ensure that IC-D can successfully process data 1 carried in the RDMA message after receiving it, PE-S1 and PE-D can submit transmission requests synchronously, so that PE-S1 submits a send request only after PE-D submits a receive request. For example, the two can perform scheduling synchronization. After submitting a receive request, PE-D sends a notification message to PE-S1. After receiving the notification message, PE-S1 submits a send request. In this way, after receiving data 1, IC-D can query RQE in RQ, read data 0 from addr0 described by RQE, perform a combined operation on data 0 and data 1, and write the result (i.e., data 1') to DRAM.

[0114] However, assuming PE-S1 has already prepared Data 1 before receiving the notification message, PE-S1 needs to wait for the notification message before submitting a send request to send Data 1. Therefore, after receiving the receive request, IC-D needs to complete the notification and transmission of Data 1 before it can receive Data 1. This results in a longer delay in receiving Data 1, as IC-D needs to combine the received Data 1 with Data 0 read in accordance with the receive request. Therefore, the longer delay in receiving Data 1 increases the latency of collective communication and reduces its efficiency.

[0115] This application proposes that PE-S1 and PE-D submit transmission requests using a synchronization-free, or asynchronous, execution mode. That is, PE-S1's submission of a transmit request is independent of the submission of a receive request. PE-S1 can submit a transmit request regardless of whether PE-D has submitted a receive request. This allows PE-S1 to prepare data before PE-D submits a receive request. This allows IC-D to receive data 1 with minimal latency after receiving the receive request submitted by PE-D, potentially even simultaneously with the receive request, thereby reducing latency in collective communication.

[0116] When IC-D receives a message carrying data 1, if PE-D has not submitted a receive request, IC-D can discard the message and send a message to IC-S1 to indicate a reception failure or to instruct retransmission. The following text takes the receive not ready (RNR) message (RNR for short) as an example. After receiving the RNR, IC-S1 can retransmit the message immediately or after a period of silence. This is beneficial for IC-D to receive a receive request before receiving one or more retransmitted messages, so that when receiving the retransmitted message, it can query the receive request, read the local data (i.e., data 0) from addr0 described in the receive request, and perform integration operations on the data 1 in the retransmitted message and the local data 0, thereby solving the problem caused by asynchrony.

[0117] This application does not limit the message type used by IC-D to notify IC-S1 to retransmit the message. As long as IC-S1 receives the message, it can retransmit the message.

[0118] Alternatively, the present application proposes that after receiving an RDMA message, if PE-D cannot obtain the corresponding receive request, it can cache the RDMA message. When it obtains the receive request, it can perform computing operations on the data 1 in the RDMA message and the local data indicated by the receive request (i.e., data 0).

[0119] The following describes an example of the communication method of the present application based on the solution of the present application described above. Figure 5-1 schematically shows an example of a possible method. The transmission element (TE) shown in Figure 5-1 can be understood as the IC described above, or as including the IC and HA in the same node described above. As shown in Figure 5-1, the method provided by the present application may include S501 to S514.

[0120] S501, TE-S1, and TE-D establish a connection between SQ and RQ. This connection is used to transmit data from PE-S1 to PE-D.

[0121] TE-S1 and TE-D can establish a connection for transmitting data from PE-S1 to PE-D. This application refers to the connection used to transmit data from PE-S1 to PE-D as connection 1, and the data transmitted from PE-S1 to PE-D as data for connection 1. Establishing connection 1 between TE-S1 and TE-D may mean that TE-S1 and TE-D obtain / negotiate the information required to transmit the data for connection 1 (referred to as connection establishment information), and then transmit the data from PE-S1 to PE-D based on the connection establishment information.

[0122] Taking into account that TE-S1 may establish connections with other TEs other than TE-D, in order to facilitate TE-S1 to send the data of connection 1 to TE-D, the link establishment information obtained by TE-S1 may include the identifier of TE-D. This application does not limit the type of TE-D identifier, as long as TE-S1 can send the data of the connection to TE-D based on the identifier of TE-D. Exemplarily, the identifier of TE-D can be the globally unique identifier (global identifier, GID) or the internet protocol address (internet protocol address, IP) address of TE-D.

[0123] Optionally, TE-S1 and TE-D can transmit data for connection 1 using the SQ and RQ. Accordingly, during the process of establishing connection 1 between TE-S1 and TE-D, TE-S1 creates an SQ for connection 1 (referred to as the SQ for connection 1). This SQ is used to store a send request instructing to send data for connection 1. TE-D creates an RQ for connection 1 (referred to as the RQ for connection 1). This RQ is used to store a receive request instructing to receive data for connection 1. For ease of description, the SQ and RQ for connection 1 are referred to as SQ1 and RQ1, respectively.

[0124] Considering that TE-S1 may create other SQs for other connections, and TE-S may create other RQs for other connections, to facilitate TE-S1 to accurately identify SQ1 and RQ1 created for connection 1, the link establishment information obtained by TE-S1 may include queue information for connection 1, which may indicate the identifiers of SQ1 and RQ1. TE-S1 may then save the queue information for connection 1 in a queue pair context (QPC).

[0125] To facilitate the TE-D to accurately identify SQ1 and RQ1 created for the connection 1, the link establishment information obtained by the TE-D may include queue information of the connection 1. The TE-D may then save the queue information of the connection 1 in a queue pair context (QPC).

[0126] Optionally, to facilitate TE-S1 and TE-D to accurately identify the link establishment information of connection 1, TE-S1 and TE-D may associate and store the link establishment information of connection 1 with the identifier of connection 1. This application does not limit the type of the identifier of connection 1. For example, connection 1 may include the identifier of PE-S1 and the identifier of PE-D.

[0127] S502, PE-S1 writes data 1 into DRAM-S1;

[0128] PE-S1 can write data 1 to DRAM-S1, assuming the write location is addr1. For example, data 1 can be the gradient of the loss function obtained by training the AI ​​model using training data by PE-S1.

[0129] S503: PE-S1 submits SR1 to TE-S1. SR1 is used to request that data 1 be sent to PE-D.

[0130] After writing data 1 to addr1, PE-S1 can submit an SR (denoted as SR1) to TE-S1. SR1 is used to request that data 1 be sent to PE-D. For example, SR1 includes the identifier of connection 1 and the address information of addr1.

[0131] S504, TE-S1 creates SQE1 in SQ1 based on SR1;

[0132] After receiving SR1, TE-S1 can determine that the SQ of connection 1 is SQ1 according to the link establishment information of connection 1, and then create an SQE (denoted as SQE1) in SQ1. SQE1 can describe addr1 or point to data 1 in addr1.

[0133] S505. When TE-S1 is dispatched to SQE1, the data 1 pointed to by SQE1 is encapsulated into message 1.

[0134] When TE-S1 is dispatched to SQE1, for example, when the pointer CI of SQ1 points to SQE1, TE-S1 can encapsulate the data 1 pointed to by SQE1 into a message (referred to as message 1). The pointer CI of SQ1 is used to point to the SQE to be consumed in SQ1.

[0135] S506. TE-S1 sends packet 1 to TE-D.

[0136] After receiving packet 1, TE-S1 can send packet 1 to TE-D. The destination address of packet 1 is TE-D's address. As described above, TE-S1 can determine the destination address of packet 1 based on the link establishment information of connection 1.

[0137] This application does not limit the timing between steps S502 and S501, as long as both S501 and S502 are executed before S503. In addition, step S502 is optional, and this application does not limit the source of data 1 in addr1. For example, data 1 can be written into addr1 by other processing units other than PE-S1.

[0138] S507, PE-D writes data 0 to addr0 in DRAM-D;

[0139] PE-D can write data 0 into DRAM-D, assuming the write location is addr0. For example, data 0 can be the gradient of the loss function obtained by PE-D using training data to train the AI ​​model.

[0140] S508, PE-D submits RR1 to TE-D, RR1 is used to describe addr0 in DRAM-D;

[0141] Based on PE-D running to the instruction to perform the calculation operation, such as "data N = data 0 + data 1", and data 0 is ready (that is, data 0 is saved in addr0), PE-D can submit RR (denoted as RR1) to TE-D. RR1 may include the identifier of connection 1 and the address information describing / indexing addr0. RR1 is used to request to receive data 1 and perform calculation operations on data 1 and the data in addr0 (that is, data 0).

[0142] S509. TE-D creates RQE1 in RQ1 based on RR1.

[0143] After receiving RR1, TE-D may determine that the RQ of connection 1 is RQ1 according to the link establishment information of connection 1, and then create an RQE (denoted as RQE1) in RQ1. RQE1 may include the address information of addr0.

[0144] Optionally, RQE1 may include indication information (or indication field) 1, where indication information 1 is used to indicate that after receiving data 1, data (ie, data 0) is read from addr0 described by RQE1, and calculation operations are performed on data 1 and data 0.

[0145] Alternatively, optionally, message 1 may also be encapsulated with indication information 1. After TE-D receives message 1 and obtains RQE1 for receiving data 1 in message 1, it determines to read data (i.e., data 0) from addr0 described by RQE1 according to indication information 1, and performs calculation operations on data 1 and data 0.

[0146] The following describes how TE-D determines RQE1 for receiving data 1, or how TE-D obtains RQE1 after receiving message 1. This description is not provided here.

[0147] This application does not limit the timing between steps S507 and S501, as long as both S501 and S507 are executed before S508. In addition, step S507 is optional, and this application does not limit the source of data 0 in addr0. For example, data 0 can be written to addr0 by other processing units other than PE-D.

[0148] S510. After receiving message 1, TE-D obtains RQE1 from RQ1.

[0149] After receiving message 1, TE-D needs to obtain RQE1 from RQ1. Message 1 may contain not only data 1 but also indication information. TE-D obtains RQE1 based on the indication information in message 1.

[0150] Considering that node D may include multiple PEs, TE-D can receive and store RRs generated by multiple PEs, and Data 1 is used to transmit to PE-D. Therefore, the indication information in Message 1 can be used to determine the RR generated by PE-D from among the RRs generated by multiple PEs. Exemplarily, the indication information in Message 1 can include the identifier of Connection 1 or the identifier of PE-D. Alternatively, TE-D can create different RQs for different PEs on node D. The indication information in Message 1 can include the identifier of RQ1, and the RQE obtained by TE-D in RQ1 is the RR generated by PE-D.

[0151] Considering that PE-S1 may need to sequentially transmit n data items to PE-D, where n is a positive integer greater than 1 and data 1 is only a portion of the data for the connection, TE-S1 sequentially receives n SRs for connection 1, and TE-D sequentially receives n RRs for connection 1. TE-S1 can sequentially store n SQEs in SQ1, and TE-D can sequentially store n RQEs in RQ1. The i-th SQE in SQ1 and the i-th RQE in RQ1 are used to transmit the same data, where i is any positive integer less than or equal to n. Furthermore, TE-S1 and TE-D consume elements from the queues sequentially. In this way, TE-D can identify RQE1 to be consumed in RQ1 as the RQE for receiving data 1. As shown in Figure 5, the pointer CI of RQ1 can be used to point to the storage location of the RQE to be consumed in RQ1. TE-D can retrieve the RQE to be consumed in RQ1 from the storage location pointed to by pointer CI.

[0152] Considering that TE-S1 may have consumed other SQEs (denoted as SQE0) before consuming SQE1 in SQ1, and sent a message (denoted as message 0) according to SQE0, and message 1 may reach TE-D before message 0, at this time, when TE-D receives message 1, the RQE pointed to by the CI of RQ1 is the RQE used to receive message 0. In order to improve the accuracy of the calculation operation performed by TE-D on data 1, the indication information in message 1 can also be used to determine the RQE used to receive data 1 from RQ1. For example, the indication information in message 1 may include a sequence identifier, and the sequence identifier is used to indicate the order of SQE1 in SQ1. TE-D can determine whether the RQE to be consumed in RQ1 is used to receive data 1 in message 1 based on the sequence identifier in message 1. Assuming that the order indicated by the order identifier is j, where j is a positive integer less than or equal to n, when the RQE to be consumed in RQ1 is the j-th RQE in RQ1, or in other words, when the order of the RQE to be consumed in RQ1 is j, TE-D may determine that the RQE to be consumed in RQ1 is used to receive data 1 in message 1; otherwise, it may determine that the RQE to be consumed in RQ1 is not used to receive data 1 in message 1. When TE-D determines that the RQE to be consumed in RQ1 is not used to receive data 1 in message 1, TE-D may buffer message 1. When the RQE to be consumed in RQ1 is the j-th RQE, TE-D may use the RQE to process data 1, or TE-D may discard message 1 and notify TE-S1 to retransmit message 1.

[0153] S511, TE-D reads data 0 pointed to by RQE1;

[0154] After TE-D obtains RQE1, it can read data 0 from addr0 according to the instructions of RQE1.

[0155] S512 , TE-D performs a calculation operation on the read data 0 and data 1 in message 1 to obtain a calculation result;

[0156] After TE-D reads data 0, it can perform calculation operations on the read data 0 and data 1 in message 1 according to the instructions of RQE1 to obtain the calculation results. This application does not limit the type of calculation operation performed by TE-D on data 0 and data 1. As mentioned above, the calculation operation can be a collective communication calculation or a calculation operation in a collective communication process or algorithm. Exemplarily, the calculation operation may include the integration operation introduced above. For example, the calculation operation may include but is not limited to at least one of an add operation, a maximum value (max) operation, an and (and) operation, an or (or) operation, an exclusive or (xor) operation, and a minimum value (min) operation.

[0157] This application does not limit the manner in which the TE-D determines the type of the computing operation. Optionally, RQE1 also includes indication information 2, which indicates the type of the computing operation. After obtaining RQE1, the TE-D can perform the computing operation of the type indicated by indication information 2 on data 1 and data 0. Alternatively, message 1 further encapsulates indication information 2, and the TE-D can determine the type of the computing operation based on indication information 2 in message 1.

[0158] Optionally, before performing calculation operations on the read data 0 and data 1 in message 1, TE-D may preprocess data 0 and / or data 1. TE-D then performs calculation operations on data 1 and preprocessed data 0, or on data 0 and preprocessed data 1, or on preprocessed data 0 and preprocessed data 1. This application does not limit the type of preprocessing; for example, preprocessing may include one or more operations such as encryption, decryption, and format conversion.

[0159] S513, TE-D writes the calculation result into addr0;

[0160] After TE-D obtains the calculation result, it can save the calculation result, for example, writing the calculation result to addr0. In other words, the storage location addr0 described by the location information in RQE1 is not only used to store data 0 that TE-D needs to read, but also used to write the calculation results of data 0 and data 1.

[0161] This application does not limit the location where TE-D writes the calculation results. For example, TE-D can write the calculation results to other storage locations other than addr0 (denoted as addr2). addr2 can be located in other storage locations in DRAM-D other than addr0, or in other memories other than DRAM-D.

[0162] This application does not limit the manner in which TE-D determines addr2. For example, message 1 may encapsulate the location information describing addr2, or the QPC stores the location information of addr2, or RQE1 includes the location information of addr2.

[0163] Step S513 is an optional step. For example, after obtaining the calculation result, TE-D may not write the calculation result into the memory, but may send the calculation result to PE-D.

[0164] S514. TE-D notifies PE-D of the completion of the calculation operation.

[0165] After TE-D writes the calculation result to addr0, it can notify PE-D of the completion of the calculation operation (i.e., completion of RQE1 or RR1). This application does not limit the manner in which TE-D notifies PE-D of the completion of the calculation operation. For example, TE-D can indicate the completion of RQE1 or RR1 by reporting an interrupt to PE-D.

[0166] Step S514 is an optional step. For example, the PE-D may determine whether the RQE1 or RR1 is completed by polling the TE-D.

[0167] Step S501 is an optional step, as long as TE-S1 can transmit data 1 to TE-D and TE-D can obtain RQE1 for receiving the data 1.

[0168] S502 to S506 are the data sending process executed by node S1, and S507 to S509 are the data receiving preparation process executed by node D. These two processes are executed asynchronously, that is, this application does not limit the timing sequence between any step in the data sending process and any step in the data receiving preparation process. Therefore, in one possible implementation, when TE-D receives message 1, S509 has not been completed. Then, TE-D fails to query RQE1 in RQ1 based on message 1. Optionally, after TE-D receives message 1 and fails to query RQE1, it can cache message 1 and execute S510 after executing S509. Alternatively, optionally, in order to save cache, as described above, after TE-D receives message 1 and fails to query RQE1, it can discard message 1 and instruct TE-S1 to retransmit message 1. This is beneficial for TE-D to complete S509 before receiving one or more retransmitted messages 1, thereby obtaining RQE1 based on the received retransmitted message 1 and then performing calculation operations.

[0169] Figure 5-2 shows a communication method based on this retransmission mechanism. The method shown in Figure 5-2 differs from the method shown in Figure 5-1 in that TE-D executes step S507 only after receiving the first packet 1 sent by TE-S1 according to SQE1. Furthermore, after TE-D receives this first packet 1, the method shown in Figure 5-2 may further include steps S515 through S517. Steps S501 through S514 of the method shown in Figure 5-2 can be understood by referring to the description of the corresponding steps above and are not further described here. The following describes steps S515 through S517.

[0170] S515. When TE-D receives message 1, it cannot obtain RQE1 in RQ1.

[0171] When TE-D receives packet 1, it cannot query RQE1 in RQ1 because RQE1 has not yet been created in RQ1.

[0172] S516: The query based on RQE1 fails, and TE-D sends an RNR to TE-S1.

[0173] As described above, after TE-D receives Message 1, if it cannot obtain the RQE (i.e., RQE1) for receiving the data in Message 1 (i.e., Data 1), TE-D may send an RNR to TE-S1 to instruct TE-S1 to retransmit Message 1. As described above, this application does not limit the type of message sent by TE-D to TE-S1 to instruct retransmission, and RNR is only used as an example.

[0174] S517: TE-S1 retransmits packet 1 to TE-D.

[0175] After TE-D sends an RNR to TE-S1, TE-S1 receives the RNR. TE-S1 then retransmits packet 1 to TE-D. TE-S1 can retransmit packet 1 immediately or after a period of silence.

[0176] As shown in Figure 5-2 , before TE-D receives the retransmitted packet 1, it can complete S507 to S509. Therefore, after TE-D receives the retransmitted packet 1, it can execute S510 to S514 in sequence.

[0177] Figure 5-2 uses the example of TE-D being able to query RQE1 upon receiving the first retransmission of packet 1. Optionally, TE-D can query RQE1 only after receiving multiple retransmissions of packet 1. In other words, if S515 to S517 are referred to as a single retransmission process, after TE-D receives the first packet 1 and before S510, the method may include one or more retransmission processes. However, for different members participating in collective communication, their collective communication progress is generally consistent, meaning that the time when PE-S1 prepares data 1 is close to the time when PE-D prepares data 0. Therefore, in collective communication scenarios, the number of retransmissions is typically small. In fact, as shown in Figure 5-1, by the time data 1 reaches TE-D, TE-D has already established RQE1, which helps reduce collective communication latency and conserve transmission resources between TE-S1 and TE-D.

[0178] After receiving any message, TE-D can use the above example in RQ to determine that the data in the message needs to be processed according to the RQE in RQ by using the operation code type in the message header of the RDMA message as send. Optionally, TE-D can also determine that the data in the message needs to be processed according to the RQE in RQ when the operation code type is other semantic types. That is, TE-S1 can send data 1 through messages of other types. Or, optionally, TE-D can determine whether it is necessary to process the data in the message according to the RQE in RQ by the values ​​of other fields in the message header of the RDMA message.

[0179] The above article takes the example of PE-S1 and PE-D using bilateral semantics to request data transfer, that is, PE-S1 submits SR and PE-D submits RR to realize data transfer between the two. Optionally, the communication system can support more data transmission semantics, for example, PE-S1 can realize data transfer between the two by submitting SR or PE-D can submit RR. In order to facilitate TE-D to identify data transmission semantics, at least one of message 1, SQE1 and QPC may include indication information 3, and indication information 3 is used to indicate the type of data transmission semantics. For example, indication information 3 can be an opcode type field carried in message 1, which can indicate send, send with invalidate, send with immediate, RDMA write, RDMA write with immediate or RDMA read, etc., and this embodiment of the application is not limited to this. TE-D identifies whether to process data 1 in message 1 according to RQE1 based on this field. For example, when this field in message 1 indicates a send operation, TE-D can obtain RQE1 from RQ1 and process data 1 according to RQE1. This process can be referred to S510 to S513. When this field in message 1 indicates an RDMA write operation, TE-D can no longer obtain RQE1 from RQ1, but process data 1 in message 1 according to the instruction of message 1.

[0180] In the above, node S1 is used as the data transmitter and node D is used as the data receiver. Optionally, the roles of node S1 and node D can be interchanged. For example, node S1 can be used to receive data 0 from node D and perform calculation operations using local data 1 and the received data 0. The method flow executed when node S1 is the receiver is the same or similar to the method flow executed by node D described above, and will not be repeated here. Similarly, the method flow executed when node D is the transmitter is the same or similar to the method flow executed by node S1 described above, and will not be repeated here.

[0181] The above describes a communication method between two nodes in a communication system. Optionally, the communication system may include more nodes. The method flow executed by the other nodes is the same or similar to the method flow executed by node S1 and / or node D.

[0182] As shown in Figure 6, the communication system may further include a node S2, which may be used to send data 2 to node D. Assume that the operands of the instruction to be executed by PE-D include data 0, data 1, and data 2. For example, the instruction is used to instruct the execution of "data N = data 0 + data 1 + data 2". After PE-D prepares data 0, it may generate RR1 and RR2 respectively. RR1 is used to instruct the reception of data 1, and an addition operation is performed on data 0 and data 1 to obtain a calculation result (recorded as data 1'). RR2 is used to instruct the reception of data 2, and an addition operation is performed on data 1' and data 2 to obtain a calculation result (recorded as data N). As shown in Figure 6, another method example provided by the present application may include S601 to S622.

[0183] S601, TE-D and node S1 establish connection 1;

[0184] S602, TE-D and node S2 establish connection 2;

[0185] The process of establishing a connection between TE-D and a node can be understood with reference to S501. For example, the process of establishing connection 1 between TE-D and node S1 may include the two parties negotiating the link establishment information of connection 1, and TE-D creating an RQ for connection 1 (denoted as RQ1). The process of establishing connection 2 between TE-D and node S2 may include the two parties negotiating the link establishment information of connection 2, and TE-D creating an RQ for connection 2 (denoted as RQ2).

[0186] When node S1 includes multiple PEs, "establishing connection 1 between TE-D and node S1" may refer to establishing connection 1 between TE-D and a PE in node S1 (for example, PE-S1). Similarly, when node S2 includes multiple PEs, "establishing connection 2 between TE-D and node S2" may refer to establishing connection 2 between TE-D and a PE in node S2 (for example, PE-S2).

[0187] This application does not limit the time sequence or order between S601 and S602.

[0188] S603, PE-D writes data 0 to addr0 in DRAM-D;

[0189] S604. PE-D submits RR1 to TE-D. RR1 includes address information describing addr0.

[0190] S605. TE-D creates RQE1 in RQ1 based on RR1.

[0191] S603 to S605 can be understood with reference to S507 to S509 and will not be described in detail here.

[0192] S606: Node S2 sends message 2 to TE-D.

[0193] After node S2 prepares data 2, it can send data 2 to TE-D via message 2. S606 can be understood with reference to S503 to S506 in Figure 5-1 and will not be described in detail here.

[0194] S607: TE-D cannot obtain the RQE (denoted as RQE2) for receiving data 2 in message 2.

[0195] After receiving Message 2, TE-D can query RQE2, which is used to receive Data 2 in Message 2. RQE2 can be RR2 stored in RQ2. As previously mentioned, RR2 is used to perform calculations on Data 2 and Data 1'. Because Data 1' is not yet ready, PE-D can temporarily not generate RR2. Consequently, TE-D cannot obtain RQE2 in RQ2.

[0196] The way in which TE-D identifies RQE2 can be understood by referring to the way in which TE-D identifies RQE1 mentioned above, and will not be repeated here.

[0197] S608: TE-D discards packet 2 and sends an RNR to node S2.

[0198] S609, node S2 retransmits message 2 according to the instruction of RNR;

[0199] S607 to S609 can be understood with reference to the retransmission process of S515 to S517 described above. For example, after TE-D receives message 2, TE-D and node S3 can perform one or more retransmission processes, which will not be described in detail.

[0200] S610: Node S1 sends message 1 to TE-D.

[0201] After node S1 prepares data 1, it can send data 1 to TE-D via message 1. S610 can be understood with reference to S503 to S506.

[0202] S611, TE-D obtains RQE1 in RQ1;

[0203] Because the RR generated by PE-D includes requests for receiving data from different senders, to facilitate distinguishing between different senders, the indication information in message 1 described above can also be used to determine the receive request for receiving data from node S1 from the receive request generated by PE-D. For example, the indication information in message 1 includes the identifier of RQ1 or the identifier of PE-S1.

[0204] S612, TE-D reads data 0 from addr0 according to the instruction of RQE1;

[0205] S613, TE-D performs an addition operation on data 0 and data 1 to obtain a calculation result (i.e., data 1');

[0206] S614, TE-D writes data 1' into addr0;

[0207] S615. TE-D notifies PE-D of the completion of RR1.

[0208] S611 to S614 can be understood with reference to the above S510 to S514 and will not be described in detail here.

[0209] S616, PE-D submits RR2 to TE-D, RR2 describes addr0 in DRAM-D;

[0210] After S615, PE-D determines that data 1' is ready and can submit RR2, described above, to TE-D. RR2 describes addr0 in DRAM-D. Optionally, as described in S513, TE-D can write data 1' to addr2. In this case, RR2 can include address information describing addr2 instead of addr0.

[0211] S617. TE-D creates RQE2 in RQ2 based on RR2.

[0212] S616 and S617 can be understood with reference to S508 and S509.

[0213] S618, TE-D obtains RQE2 in RQ2;

[0214] Assume that TE-D executes S617 before receiving the message 2 retransmitted once or multiple times by the node S2. Then, after receiving the message 2, TE-D can query RQ2 to obtain RQE2.

[0215] Because the RR generated by PE-D includes requests for receiving data from different senders, to facilitate distinguishing between different senders, the indication information in message 2 can also be used to determine the receive request for receiving data from node S2 from the receive requests generated by PE-D. For example, the indication information in message 2 includes the identifier of RQ2 or the identifier of the PE in node 2.

[0216] S619, TE-D reads data 1' from addr0 according to the instruction of RQE2;

[0217] S620, TE-D performs an addition operation on data 1' and data 2 to obtain a calculation result (ie, data N);

[0218] S621, TE-D writes data N into addr0;

[0219] S622: TE-D notifies PE-D of the completion of RR2.

[0220] S618 to S622 can be understood with reference to S510 to S514 and will not be described in detail here.

[0221] As shown in Figure 6, for node D in the system to receive data, different nodes in the system can correspond to different RQs. After receiving the RR for receiving data sent by a node or a PE on the node, the TE-D creates an RQE in the RQ corresponding to that node. Furthermore, after receiving a message from that node or PE, it queries the RQE in the RQ corresponding to that node. This way, as shown in Figure 6, even if message 2 arrives at message 1 before message 1, it still helps ensure that the TE-D calculates each operand according to the order of operations indicated by the instruction. This helps ensure correct execution of the instruction even when multiple computational operations in the instruction do not satisfy the commutative law.

[0222] Node S1's execution of S610 and node S2's execution of S606 are performed asynchronously. That is, this application does not limit the timing between S610 and S606. S603-S605 are node D's preparation process for receiving data 1. As previously described, S610 and this process are performed asynchronously, and S606 and this process are also performed asynchronously. S616-S617 are node D's preparation process for receiving data 2. As previously described, S606 and this process are performed asynchronously.

[0223] Assume that the instruction to be executed by PE-D still instructs to execute "data N = data 0 + data 1 + data 2". However, considering that the calculation operation in the instruction satisfies the commutative law, if data 2 arrives at TE-D before data 1, TE-D first calculates data 0 and data 2 to obtain the calculation result (recorded as data 2'). Then, when data 1 arrives, data 1 and data 2' are calculated to obtain the calculation result (i.e., data N). This is conducive to improving the calculation efficiency of collective communication. In order to achieve the purpose of first-come-first-served, the present application proposes that TE-D puts RR1 and RR2 into the same RQ. After PE-D prepares data 0, it puts RR1 and RR2 into the same RQ, which is conducive to RR1 being used to receive data 1 or data 2. Similarly, RR2 is used to receive data 1 or data 2, so that when data 2 arrives before data 1, TE-D uses RR1 to calculate data 2, and then uses RR2 to calculate data 1, thereby reducing the instruction delay and improving the efficiency of collective communication. Based on this concept, as shown in FIG7 , another method example provided by this application may include S701 to S719 .

[0224] S701, TE-D and node S1 establish connection 1;

[0225] S702, TE-D and node S2 establish connection 2;

[0226] The process of establishing a connection between a TE-D and a node can be understood with reference to S501. For example, the process of establishing connection 1 between a TE-D and node S1 may include the two parties negotiating the link establishment information for connection 1, and TE-D creating an RQ for connection 1. The process of establishing connection 2 between a TE-D and node S2 may include the two parties negotiating the link establishment information for connection 2, and TE-D creating an RQ for connection 2. As described above, RR1 and RR2 share the same RQ, so TE-D creates the same RQ (denoted as SRQ) for connections 1 and 2.

[0227] When node S1 includes multiple PEs, "establishing connection 1 between TE-D and node S1" may refer to establishing connection 1 between TE-D and a PE in node S1 (for example, PE-S1). Similarly, when node S2 includes multiple PEs, "establishing connection 2 between TE-D and node S2" may refer to establishing connection 2 between TE-D and a PE in node S2 (for example, PE-S2).

[0228] This application does not limit the time sequence or order between S701 and S702.

[0229] S703, PE-D writes data 0 to addr0 in DRAM-D;

[0230] S704. PE-D submits RR1 to TE-D. RR1 includes address information describing addr0.

[0231] S705. TE-D creates RQE1 in SRQ based on RR1.

[0232] S703 to S705 can be understood with reference to S507 to S509 and will not be described in detail here.

[0233] S706. Node S2 sends message 2 to TE-D.

[0234] S706 can be understood with reference to S503 to S506 and S606, and will not be described in detail here.

[0235] S707, TE-D obtains RQE1 in SRQ;

[0236] As previously mentioned, connection 1 and connection 2 can share the same SRQ. Therefore, after TE-D receives data 2 from connection 2, it can obtain RQE1 from the SRQ. In other words, TE-D does not need to distinguish whether RQE1 is used to receive data 1 from connection 1 or data 2 from connection 2. In other words, unlike S618 where TE-D queries RQE2 based on the indication information in message 2, the indication information in message 2 can not be used to determine the receive request for receiving data from node S2 from the receive request generated by PE-D.

[0237] S708, TE-D reads data 0 from addr0 according to the instruction of RQE1;

[0238] S709, TE-D performs an addition operation on data 0 and data 2 to obtain a calculation result (i.e., data 2');

[0239] S710, TE-D writes data 2' into addr0;

[0240] S711. TE-D notifies PE-D of the completion of RR1.

[0241] S707 to S711 can be understood by referring to the above S510 to S514, and will not be repeated here.

[0242] S712, PE-D submits RR2 to TE-D, RR2 describes addr0 in DRAM-D;

[0243] After S711, PE-D determines that the calculation result of a calculation operation instructed by the instruction has been written into addr0, and can submit RR2 to TE-D. RR2 describes addr0 in DRAM-D.

[0244] S713. TE-D creates RQE2 in SRQ based on RR2.

[0245] S712 and S713 can be understood with reference to S508 and S509.

[0246] S714. Node S1 sends message 1 to TE-D.

[0247] After node S1 prepares data 1, it can send data 1 to TE-D via message 1. S714 can be understood with reference to S503 to S506.

[0248] S715, TE-D obtains RQE2 in SRQ;

[0249] As previously mentioned, connection 1 and connection 2 can share the same SRQ. Therefore, after TE-D receives data 1 from connection 1, it can obtain the RQE in the SRQ. Since data 1 arrives at TE-D later than data 2, the RQE obtained by TE-D for data 1 is RQE2. In other words, TE-D does not need to distinguish whether RQE2 is used to receive data 1 from connection 1 or data 2 from connection 2. Unlike in S611, where TE-D queries RQE1 based on the indication information in message 1, the indication information in message 1 can be used to determine the receive request for receiving data from node S1 from the receive request generated by PE-D.

[0250] S716. TE-D reads data 2' from addr0 according to the instruction of RQE2.

[0251] S717, TE-D performs an addition operation on data 2' and data 1 to obtain a calculation result (i.e., data N);

[0252] S718, TE-D writes data N into addr0;

[0253] S719: TE-D notifies PE-D of the completion of RR2.

[0254] S715 to S719 can be understood with reference to S510 to S514 and will not be described in detail here.

[0255] When the computing operation satisfies the commutative law, the present application proposes that for the node D to receive data in the system, other different nodes in the system correspond to the same RQ, and the present application refers to the RQ as a shared receive queue (SRQ). After TE-D receives the RR, no matter which node the RR is used to receive data from, TE-D creates an RQE in the SRQ, and after receiving the message, no matter which node the message comes from, TE-D queries the RQE in the SRQ. It is beneficial for TE-D to perform computing operations immediately after the message sent by any node reaches TE-D, thereby reducing the delay for TE-D to complete the computing operation and improving the execution efficiency of collective communication.

[0256] The SQ and RQ methods described above are only one possible way for the TE to store and query requests generated by the PE. In practice, the TE can use other methods to store requests generated by the PE and query the corresponding requests after receiving the messages. In Figure 5-1, S504 and S505 are optional and can be replaced by other steps to store and consume SR1. Similarly, S509 and S510 are optional and can be replaced by other steps to store and consume RR1.

[0257] The above describes the communication method and system provided by this application using the collective communication scenario as an example. This application does not limit the application scenario of this application solution, and accordingly, this application does not limit the computing operation to be performed by node D to the computing operation in collective communication.

[0258] The preceding example uses an addition operation between the local data of node D and the received data as an example. In actual applications, the addition operation can be replaced with another type of operation. The preceding example uses the same type of operation performed on data 1 and data 2 as an example. In actual applications, the operations performed on the two data can be different types of operations.

[0259] The above example uses PE-D writing data 0 into DRAM-D and PE-S1 writing data 1 into DRAM-S1 as an example. In actual applications, data can be written into DRAM by other devices (such as TE).

[0260] It should be understood that in the various method examples of this application, the order of the sequence numbers of the above steps does not imply the order of execution. The order of execution of each step should be determined by its function and internal logic. It should be understood that the terms used in this manner are interchangeable where appropriate. This is merely a way of distinguishing objects with the same attributes when describing them in this application.

[0261] In the examples of this application, various numerical numbers are only used for the convenience of description and are not used to limit the scope of the examples of this application.

[0262] In the system provided in the present application, the present application does not limit the deployment mode of each unit in the node. For example, all or part of the units in the same node can be integrated into the same chip. As shown in Figure 5-1 or Figure 5-2 or Figure 6 or Figure 7, the PE and TE of the same node can be integrated into the same processor, or the PE and TE can be physically separated. For example, the PE is a processor or is integrated into the processor, and the TE is set outside the processor, for example, in an independent network card. In the system shown in Figure 4, the PE, HA, IC and MC in the same node can be integrated into the same processor, or the PE, HA and MC in the same node can be integrated into the same processor, and the IC is an independent network card or is set in an independent network card.

[0263] This application does not limit the implementation manner of the TE performing the above-mentioned corresponding method steps. The above-mentioned corresponding method steps include, for example, the method steps performed by TE-D in Figure 5-1 or Figure 5-2 or Figure 6 or Figure 7.

[0264] Optionally, the TE may execute the above-mentioned corresponding method steps in a software manner. For example, the TE may include a processor and a memory, wherein the memory stores instructions, and the processor executes the above-mentioned corresponding method steps by executing the instructions.

[0265] Alternatively, the TE may perform the above-mentioned corresponding method steps in hardware. For example, the TE may include one or more circuits configured to perform the above-mentioned corresponding method steps. The one or more circuits may be integrated into the same chip or different chips.

[0266] Optionally, the TE may perform the above-mentioned corresponding method steps through a combination of software and hardware. For example, the TE may include a processor, a memory, and one or more circuits. The processor executes instructions in the memory to perform a portion of the above-mentioned corresponding method steps, and the one or more circuits are used to perform the remaining steps of the above-mentioned corresponding method steps.

[0267] That is to say, those skilled in the art can implement the units and algorithm steps of the examples disclosed herein in electronic hardware, computer software, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0268] The above describes the method provided by the present application. The following describes the device provided by the present application. Figure 8 schematically illustrates a possible structure of a communication device. As shown in Figure 8, the communication device may include a communication module and a processing module.

[0269] The communication module is configured to receive a message sent by a first node, the message encapsulating first data to be transmitted to a first processing unit among one or more processing units. The processing module is configured to obtain a first receive request generated by the first processing unit, the first receive request including first address information describing a first storage location. The processing module is further configured to read second data from the first storage location according to the first address information, and perform a calculation operation on the first data and the second data to obtain a calculation result.

[0270] Optionally, the message also encapsulates first indication information, and the first indication information is used to determine the receiving request generated by the first processing unit from the receiving requests generated by the one or more processing units; the transmission unit is specifically used to obtain the first receiving request according to the first indication information in the message.

[0271] Optionally, the communication system also includes other nodes in addition to the first node and the second node, and the receiving request generated by the first processing unit includes a receiving request for receiving data from the first node and a receiving request for receiving data from the other nodes, and the first indication information is also used to determine the receiving request for receiving data from the first node from the receiving request generated by the first processing unit.

[0272] Optionally, the first data is a part of the target data transmitted by the first node to the first processing unit, and the receiving request generated by the first processing unit for receiving data from the first node includes a receiving request for receiving the first data and a receiving request for receiving other data in the target data except the first data, and the first indication information is also used to determine the receiving request for receiving the first data from the receiving request generated by the first processing unit for receiving data from the first node.

[0273] Optionally, the processing module is further configured to notify the processing unit of the completion of the calculation operation after obtaining the calculation result.

[0274] The first receiving request further includes second indication information, where the second indication information is used to instruct to perform the computing operation on the second data and the first data.

[0275] Optionally, the first receiving request further includes third indication information, and the third indication information is used to indicate the type of the computing operation.

[0276] Optionally, the processing module is further configured to pre-process the first data and / or the second data before performing the computing operation on the first data and the second data.

[0277] Optionally, the first receiving request also includes second address information describing a second storage location, where the second storage location is used to store the calculation result; the processing module is further used to write the calculation result into the second storage location according to the second address information.

[0278] Optionally, the second transmission unit is further used to write the calculation result into the first storage location according to the first address information.

[0279] Optionally, the message is a Remote Direct Memory Access protocol RDMA message or a Transmission Control Protocol TCP message.

[0280] Optionally, the computing operation is collective communication computing.

[0281] The communication device may be the TE-D or node D described above. The message may be message 1 or message 2, the first processing unit may be, for example, PE-D, and the first node may be, for example, node S1 or node S2 or PE-S1 in node S1. The first data may be data 1, and the second data may be data 0 or data 2'. Alternatively, the first data may be data 2, and the second data may be data 0 or data 1'. The first receive request may be, for example, RR1 (or RQE1) or RR2 (or RQE2), the first storage location may be addr0, and the second storage location may be addr2, and the calculation operation may include any one or more operations described above. The first indication information may be the indication information in the message described above, the second indication information may be the indication information 1 described above, and the third indication information may be the indication information 2 described above.

[0282] For example, the communication module can be used to execute S506, and the processing module can be used to execute all or part of steps S508 to S514. Optionally, the communication module is also used to execute all or part of steps S516 and S517, and the processing module is also used to execute S515.

[0283] For example, the communication module may be used to execute all or part of steps S606, S608, and S609, and the processing module may be used to execute all or part of steps S604-S605, S607, and S611-S622.

[0284] For example, the communication module may be used to execute S706 and / or S714, and the processing module may be used to execute all or part of steps S704 to S705, S707 to S713, and S715 to S719.

[0285] As described above, the modules shown in FIG8 may be virtual function modules generated by the processor by executing instructions in the memory, or the modules shown in FIG8 may be hardware circuits, or some of the modules shown in FIG8 may be virtual function modules generated by the memory by executing instructions in the memory, and the remaining modules may be hardware circuits.

[0286] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the division of the modules in the device described in Figure 8 is merely schematic and can be understood as a logical functional division. In actual implementation, other division methods may be used. For example, a module can be split into multiple units, multiple modules can be integrated together or integrated into another system, or some features can be ignored or not executed.

[0287] When using software to realize the various aspects of the embodiment of the application, or the possible implementation of various aspects, the above-mentioned various aspects, or the possible implementation of various aspects can be realized in whole or in part in the form of a computer program product. A computer program product refers to instructions (or computer-readable instructions or computer program instructions or functional programs or program codes) stored in a computer-readable medium. When loading and executing these instructions on a computer, the process or function described in the embodiment of the application are generated in whole or in part.

[0288] The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof. For example, the computer-readable storage medium may be a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable compact disc read-only memory (CD-ROM).

[0289] The terms "including" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent to the process, method, product, or apparatus. The term "plurality" appearing in the embodiments of the present application refers to two or more.

[0290] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A communication system, characterized in that: The communication system includes a first node and a second node, the second node includes a transmission unit and one or more processing units, the one or more processing units include a first processing unit; The first node is configured to send a message to the transmission unit, where the message encapsulates first data to be transmitted to the first processing unit; The transmission unit is configured to receive the message and obtain a first receiving request generated by the first processing unit, wherein the first receiving request includes first address information describing a first storage location; The transmission unit is further configured to read the second data in the first storage location according to the first address information, and perform a calculation operation on the first data and the second data to obtain a calculation result.

2. The communication system according to claim 1, wherein: The message further encapsulates first indication information, where the first indication information is used to determine the reception request generated by the first processing unit from the reception requests generated by the one or more processing units; The transmission unit is specifically configured to obtain the first receiving request according to the first indication information in the message.

3. The communication system according to claim 2, wherein: The communication system also includes other nodes in addition to the first node and the second node. The receiving request generated by the first processing unit includes a receiving request for receiving data from the first node and a receiving request for receiving data from the other nodes. The first indication information is also used to determine the receiving request for receiving data from the first node from the receiving request generated by the first processing unit.

4. The communication system according to claim 3, wherein: The first data is a part of the target data transmitted by the first node to the first processing unit. The receiving request generated by the first processing unit for receiving data from the first node includes a receiving request for receiving the first data and a receiving request for receiving other data in the target data except the first data. The first indication information is also used to determine the receiving request for receiving the first data from the receiving request generated by the first processing unit for receiving data from the first node.

5. The communication system according to any one of claims 1 to 4, characterized in that: The first receiving request further includes second indication information, where the second indication information is used to instruct to perform the computing operation on the second data and the first data.

6. The communication system according to any one of claims 1 to 5, characterized in that: The first receiving request further includes third indication information, where the third indication information is used to indicate a type of the computing operation.

7. The communication system according to any one of claims 1 to 6, characterized in that: The transmission unit is further configured to pre-process the first data and / or the second data before performing the computing operation on the first data and the second data.

8. The communication system according to any one of claims 1 to 7, characterized in that: The first receiving request further includes second address information describing a second storage location, where the second storage location is used to store the calculation result; The transmission unit is further configured to write the calculation result into the second storage location according to the second address information.

9. The communication system according to any one of claims 1 to 8, characterized in that: The transmission unit is further configured to write the calculation result into the first storage location according to the first address information.

10. The communication system according to any one of claims 1 to 9, characterized in that: The message is a Remote Direct Memory Access protocol RDMA message or a Transmission Control Protocol TCP message.

11. The communication system according to any one of claims 1 to 10, characterized in that: The computing operation is collective communication computing.

12. A communication method, characterized in that: The communication method is applied to a communication system, the communication system including a first node and a second node, the second node including a transmission unit and one or more processing units, the one or more processing units including a first processing unit, the method including: The first node sends a message to the transmission unit, where the message encapsulates first data to be transmitted to the first processing unit; The transmission unit receives the message and obtains a first receiving request generated by the first processing unit, where the first receiving request includes first address information describing a first storage location; The transmission unit reads the second data in the first storage location according to the first address information, and performs a calculation operation on the first data and the second data to obtain a calculation result.

13. A communication method, characterized in that: The method comprises: receiving a message sent by a first node, wherein the message encapsulates first data to be transmitted to a first processing unit among the one or more processing units; Obtaining a first receiving request generated by the first processing unit, where the first receiving request includes first address information describing a first storage location; The second data in the first storage location is read according to the first address information, and a calculation operation is performed on the first data and the second data to obtain a calculation result.

14. The method according to claim 13, characterized in that The message further encapsulates first indication information, where the first indication information is used to determine the reception request generated by the first processing unit from the reception requests generated by the one or more processing units, and obtaining the first reception request generated by the first processing unit includes: Obtain the first receiving request according to the first indication information in the message.

15. The method according to claim 14, characterized in that The receiving request generated by the first processing unit includes a receiving request for receiving data from the first node and a receiving request for receiving data from other nodes other than the first node and the node where the one or more processing units are located. The first indication information is also used to determine the receiving request for receiving data from the first node from the receiving request generated by the first processing unit.

16. The method according to claim 15, characterized in that The first data is a part of the target data transmitted by the first node to the first processing unit. The receiving request generated by the first processing unit for receiving data from the first node includes a receiving request for receiving the first data and a receiving request for receiving other data in the target data except the first data. The first indication information is also used to determine the receiving request for receiving the first data from the receiving request generated by the first processing unit for receiving data from the first node.

17. The method according to any one of claims 13 to 16, characterized in that The first receiving request further includes second indication information, where the second indication information is used to instruct to perform the computing operation on the second data and the first data.

18. The method according to any one of claims 13 to 17, characterized in that The message is a Remote Direct Memory Access protocol RDMA message or a Transmission Control Protocol TCP message.

19. The method according to any one of claims 13 to 18, characterized in that The computing operation is collective communication computing.

20. The method according to any one of claims 13 to 19, characterized in that The one or more processing units are deployed in a second node other than the first node, and the method is executed by the second node or by a transmission unit in the second node.

21. A communication device, characterized in that: The communication device comprises: a communication module, configured to receive a message sent by a first node, wherein the message encapsulates first data to be transmitted to a first processing unit among the one or more processing units; a processing module, configured to obtain a first receiving request generated by the first processing unit, wherein the first receiving request includes first address information describing a first storage location; The processing module is further configured to read the second data in the first storage location according to the first address information, and perform a calculation operation on the first data and the second data to obtain a calculation result.

22. The communication device according to claim 21, wherein: The one or more processing units are deployed on a second node other than the first node, and the communication device is the second node or is deployed on the second node.

23. The communication device according to claim 22, wherein: The communication device is a chip, a network interface controller, or a host device.

24. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, and when the program code is executed by a processor in a computer device, the method according to any one of claims 13 to 20 is implemented.

25. A computer program product, characterized in that When the program code contained in the computer program product is executed by a processor in a computer device, the method according to any one of claims 13 to 20 is implemented.

Images