Data calculation method and device, equipment and storage medium

By performing aggregation calculations in the network adapter of distributed clusters, the problems of resource waste and delay in reduce operations in distributed training are solved, and the computing efficiency and response speed are improved.

CN119065833APending Publication Date: 2024-12-03DOUYIN VISION CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411019374.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In distributed training scenarios, the reception, processing and computing process of data in reduce operations in the prior art requires the participation of the central processor (CPU) and graphics processor (GPU), resulting in waste of system resources and increased delay, affecting the calculation efficiency.

Method used

By receiving and performing aggregated computing messages in the network adapter of the second node of the distributed cluster, data is avoided from being processed by the CPU or GPU during transmission, data reception and aggregated computing are performed directly in the network adapter, and the result is written Back to the target storage address.

Benefits of technology

It improves the efficiency and speed of data processing, reduces the waiting time for data transmission and processing, reduces the delay, and improves the response speed and real-time performance of distributed clusters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065833A_ABST
    Figure CN119065833A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a data calculation method and device, equipment and a storage medium. The method comprises the steps that at a second node in a distributed cluster, a first message sent by a first node in the distributed cluster is received through a network adapter of the second node, the first message indicates a first primitive operation, and the first primitive operation indicates execution of aggregation calculation on data of the first node and data of the second node; in response to the first message, executing a first primitive operation on the data of the first node and the data of the second node through the network adapter to obtain an aggregation calculation result; and writing the result of the aggregation calculation back to the target storage address of the second node. Therefore, by executing data receiving and aggregation calculation operation in the network adapter, additional processing of the data through the central processing unit or the graphics processing unit in the transmission process is avoided, and the efficiency and speed of distributed cluster data processing are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of information technology, and in particular, to data computing methods, devices, equipment, and computer-readable storage media. Background Art

[0002] In machine learning and high-performance computing scenarios, especially when computing on large data sets, it is usually necessary to use a distributed system. In this distributed computing scenario, collective communication operations are required between nodes to achieve data exchange and computing between nodes. These collective communication operations include various collective communication primitive operations.

[0003] Among them, the reduce operation is a commonly used data communication and computing operation, which is used to aggregate the local data calculated by each node to obtain global parameters or indicators. In related technologies, the data reception, processing and calculation process in the reduce operation usually requires the participation of processing units such as the central processing unit (CPU) and the graphics processing unit (GPU), resulting in a waste of system resources and an increase in latency.

[0004] Therefore, it is worthy of attention to improve the data computing efficiency in distributed training scenarios. Summary of the invention

[0005] In a first aspect of the present disclosure, a data computing method is provided. The method comprises: receiving, at a second node in a distributed cluster, a first message sent by a first node in the distributed cluster via a network adapter of the second node, the first message indicating a first primitive operation, the first primitive operation indicating performing aggregate computing on data of the first node and data of the second node; in response to the first message, performing the first primitive operation on the data of the first node and the data of the second node via the network adapter to obtain a result of the aggregate computing; and writing the result of the aggregate computing back to a target storage address of the second node.

[0006] In a second aspect of the present disclosure, a data computing device is provided. The device includes: a receiving module configured to receive, at a second node in the distributed cluster, via a network adapter of the second node, a first message sent by a first node in the distributed cluster, the first message indicating a first primitive operation, the first primitive operation indicating the execution of aggregate calculation on the data of the first node and the data of the second node; a computing module configured to execute the first primitive operation on the data of the first node and the data of the second node through the network adapter in response to the first message, to obtain the result of the aggregate calculation; and a write-back module configured to write the result of the aggregate calculation back to the target storage address of the second node.

[0007] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0008] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0009] In a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0010] It should be understood that the content described in this content section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0012] Figure 1A A schematic diagram of a reduce operation in a distributed cluster is shown;

[0013] Figure 1B A schematic diagram of a reduce operation in a node in a machine learning scenario based on GPU computing is shown;

[0014] Figure 2A A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0015] Figure 2B A schematic diagram showing a data computing operation performed in a node according to an embodiment of the present disclosure is shown;

[0016] Figure 3A A flowchart showing a process of data calculation according to some embodiments of the present disclosure;

[0017] Figure 3B A schematic diagram of a message format of a first message according to some embodiments of the present disclosure is shown in FIG.

[0018] Figure 4AA schematic diagram of a process of performing a first primitive operation according to some embodiments of the present disclosure is shown in FIG.

[0019] Figure 4B A schematic diagram of a process of acquiring first data and second data according to some embodiments of the present disclosure is shown in FIG.

[0020] Figure 4C A schematic diagram of a process of performing aggregate calculation according to some embodiments of the present disclosure is shown in FIG.

[0021] Figure 4D A schematic diagram of a data calculation process according to some embodiments of the present disclosure is shown in FIG.

[0022] Figure 5 A block diagram showing an apparatus for data calculation according to some embodiments of the present disclosure; and

[0023] Figure 6 A block diagram of an electronic device capable of implementing one or more embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0024] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0025] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0026] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0028] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0031] As briefly mentioned above, with the rapid development of machine learning and deep learning, the parameters and data scale that need to be processed and calculated are becoming larger and larger. When performing machine learning training on large-scale data sets, the amount of data that needs to be processed is usually very large, and sometimes a single computing node may not be able to meet the computing and storage requirements. In order to speed up the training process and improve efficiency and performance, a distributed system is usually used to distribute tasks to multiple computing nodes for parallel processing.

[0032] In this distributed training scenario, each node needs to communicate and collaborate with each other to update and optimize model parameters. Each node is usually responsible for processing part of the data and calculating the corresponding local parameters. Then, in order to obtain global data, the data in each node needs to be aggregated, and this process is called the reduce operation.

[0033] In distributed training scenarios and high-performance computing scenarios, collective communication operations are often required to implement data exchange and calculation between nodes. These collective communication operations include various primitives, such as reduce, AllReduce, reduce-scatter, broadcast, gather, all-gather, scatter, and all-to-all. Among them, reduce, AllReduce, and Scatter-Reduce are collective communication primitives related to reduce operations.

[0034] Figure 1A Figure 1 shows a schematic diagram of a reduce operation in a distributed cluster. Figure 1A As shown, the reduce operation on the four nodes, node 0, node 1, node 2, and node 3 in the distributed cluster is taken as an example.

[0035] When performing these collective communication operations, the operations are usually broken down into multiple steps, each of which includes the process of receiving data and aggregate calculation. First, each node will receive data sent from another node (possibly an adjacent node). These data are usually divided into multiple blocks. For example, the data blocks of data A in different nodes are divided into A0, A1, A2, and A3. When the data is received, the received data will be transmitted to the local computing resources and aggregated with the local data. This calculation process can, for example, be to add the received data to the local data to obtain an aggregate result. This process will be repeated many times until all data blocks are aggregated to one node or the required computing task is completed.

[0036] Figure 1B FIG. 1 shows a schematic diagram of a reduce operation performed in a node 10 in a machine learning scenario based on GPU computing. Figure 1B As shown, the network adapter 10-1 receives data sent by other nodes, and after the CPU learns that the data reception is completed through polling, it notifies GPU10-2. GPU10-2 needs to obtain this data for subsequent computing operations. This data transmission process requires the control plane of GPU10-2 to participate in controlling the reception and storage of data. Among them, the network adapter 10-1 can also be called a network interface card (NIC), network card, etc.

[0037] After receiving the data, GPU 10-2 needs to extract the data from memory 10-3 and perform corresponding calculation operations, such as summation and averaging, in the calculation unit of GPU 10-2, such as streaming multiprocessor 10-4. After the calculation is completed, GPU 10-2 needs to write the calculated result back to memory 10-3. This process also requires management and scheduling by the control plane of GPU 10-2.

[0038] In the related art, the calculations performed in the GPU will have a negative impact on the performance of collective communication. For example, the throughput may decrease. This is particularly common in large model computing scenarios. In addition, in the current reduce-related collective communication primitive operations, the processing flow is usually lengthy and requires the participation of the CPU. Moreover, in the related art, although there is a way to offload part of the computing tasks to the network adapter, in this process, the current node needs to first notify the target node to send the data that needs to be aggregated and calculated, and after receiving the data, perform aggregate calculations with the local data, and then send the calculation results to the target node. This process includes multiple data sending and receiving processes, and consumes additional CPU resources, which may increase latency or cause network communication bottlenecks and waste of system resources.

[0039] Therefore, in order to solve the above problems and improve the efficiency and performance of distributed training, a more efficient data computing method is needed to reduce the overhead of data transmission and processing, thereby improving the efficiency of distributed training and high-performance computing.

[0040] In an embodiment of the present disclosure, a data computing scheme is provided. According to an embodiment of the present disclosure, at a second node in a distributed cluster, a first message sent by a first node in the distributed cluster is received via a network adapter of the second node, the first message indicates a first primitive operation, and the first primitive operation indicates performing aggregate computing on the data of the first node and the data of the second node; in response to the first message, the first primitive operation is performed on the data of the first node and the data of the second node through the network adapter to obtain a result of the aggregate computing; and the result of the aggregate computing is written back to the target storage address of the second node.

[0041] By indicating the first primitive operation in the first message, data reception and aggregation calculation operations can be performed in the network adapter, thereby avoiding additional processing of the data by the CPU or GPU during transmission, thereby improving the efficiency and speed of data processing.

[0042] In addition, moving data computing operations to the network adapter for execution reduces the waiting time for data transmission and processing, thereby reducing the latency of data processing and improving the response speed and real-time performance of the distributed cluster.

[0043] Figure 2A A schematic diagram of an example environment 100 in which an embodiment of the present disclosure can be implemented is shown. In environment 100, a distributed cluster may include several nodes 110-1, 110-2, ... 110-N (collectively or individually referred to as node 110). Each node includes a network adapter 111-1, 111-2, ... 111-N, and a memory 112-1, 112-2, ... 112-N. The process of data aggregation calculation involved in an embodiment of the present disclosure can be implemented at any two different nodes in a distributed cluster, such as at node 110-1 and node 110-2. For ease of understanding, node 110-1 and node 110-2 are used for example description in the following.

[0044] In some embodiments, at node 110-2 in the distributed cluster, a message 113-1 sent by another node 110-1 in the distributed cluster is received via the node's network adapter 111-2. Message 113-1 indicates a primitive operation, that is, an operation of performing aggregate calculation on data of node 110-1 and data of node 110-2.

[0045] In some embodiments, the memory 112 is not limited to the on-chip memory of the GPU, but can also be the internal memory of the network card or the host memory, etc. In short, the specific implementation of the memory 112 is not limited to a certain hardware configuration, but can be flexibly configured according to the actual needs of the node.

[0046] Figure 2B FIG. 1 is a schematic diagram showing a data computing operation performed in the node 110-2 according to an embodiment of the present disclosure. Figure 2B As shown, in response to message 113-1, node 110-2 can perform the primitive operation on the data of node 110-1 and the data of node 110-2 through network adapter 111-2 to obtain the result of the aggregate calculation. Node 110-2 can directly write the result of the aggregate calculation back to the target storage address corresponding to memory 112-2.

[0047] After receiving the data that needs to be aggregated and calculated from other nodes, the network adapter 111-2 obtains the data that needs to be aggregated and calculated from the memory 112-2 of the node and performs calculations, and finally writes the calculation results directly to the GPU's on-chip memory 112-2. It should be noted that this process is all completed by the network adapter 111-2, and the GPU directly receives the calculation results. The process of receiving data from other nodes and aggregate calculations does not require the participation of the GPU's control plane and stream multiprocessor.

[0048] It should be understood that the above-mentioned node 110 - 1 and node 110 - 2 are merely exemplary illustrations of two different nodes that need to perform aggregate computing operations in a distributed cluster, and are not intended to limit the present disclosure in any form.

[0049] In the environment 100, the node 110 may be any type of device with computing capabilities, including a terminal device or a server device. The terminal device may be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. The server device may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like.

[0050] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and does not imply any limitation on the scope of the present disclosure.

[0051] Figure 3AA flowchart of a process 300 of data calculation according to some embodiments of the present disclosure is shown. The process 300 can be implemented at any two different nodes in a distributed cluster, such as node 110-1 and node 110-2. For ease of understanding, the following is an example of a first node and a second node.

[0052] In box 310, at a second node in the distributed cluster, a first message sent by a first node in the distributed cluster is received via a network adapter of the second node. In some embodiments, the first message indicates a first primitive operation. The first primitive operation indicates that aggregate calculations are performed on the data of the first node and the data of the second node. For example, a new primitive operation can be configured in advance in each node in the distributed cluster to indicate the operation of completing data reception and aggregate calculations in the network adapter. Further, the first node can send a first message to the second node, instructing the second node to perform a corresponding first primitive operation to perform aggregate calculations on the data of the first node and the data of the second node.

[0053] In some embodiments, the first message includes at least one of the following: first data that needs to be aggregated and calculated in the first node, an operation code, an aggregate calculation type identifier, a storage address of second data that needs to be aggregated and calculated in the second node in the second node, and a target storage address.

[0054] The operation code indicates the first primitive operation. The aggregate calculation type identifier indicates the calculation type of the aggregate calculation of the first primitive operation, such as addition, maximum value, minimum value, average value, logical AND, logical OR, XOR, etc.

[0055] As an example, a first node sends a first message to a second node through a network adapter. When sending the first message, since the first message may be relatively large, it exceeds the maximum transmission unit (Maximum Transmission Unit, MTU) specified by the network link protocol. Therefore, when sending the first message, the network adapter may divide the message into multiple messages according to the MTU size of the network link, and include the data and operations that the first message needs to indicate in the message field to ensure smooth transmission.

[0056] For example, please refer to Figure 3B , Figure 3BA schematic diagram of the message format of the first message of some embodiments of the present disclosure is shown in . As shown in the figure, the message 340 may include a message header consisting of the following fields: an operation code field to identify the type of the message 340, which is used here to represent the first primitive operation, and can also be used to distinguish the first, middle and tail messages of the message, so as to recover and reorganize during the message transmission process; an aggregate calculation type identification field, such as addition, maximum value, minimum value, average value, logical AND, logical OR, XOR, etc. This field determines the specific calculation method used in the aggregate calculation process; identifies the total length of the message, which is used to correctly parse the message and allocate the corresponding buffer at the second node; a data type, which is used to indicate the type and size of the first data in the message payload, so as to perform correct type conversion and memory allocation when parsing the data. The message 340 may also include a data field to transmit the first data that needs to be aggregated and calculated in the first node.

[0057] In some embodiments, the message 340 also includes memory address information. For example, the memory address information may include a second data storage address and a target storage address.

[0058] In some embodiments, multiple fields in the message 340 may be merged into one field. For example, if the calculation result is still saved to the address where the node originally stores data, that is, the second data storage address and the target storage address are both located at the same memory address, the second data storage address and the target storage address may be merged into one field.

[0059] In some embodiments, the memory address information may indicate different address information according to the primitive operation indicated by the opcode. As an example, the memory address information in the message 340 may include the first data storage address. The second node may determine the first data storage address by parsing the first message, thereby reading the first data into the memory of the second node.

[0060] As another example, according to the primitive operation indicated by the operation code, the message 340 may not include memory address information. After receiving the first message, the second node provides the storage address of the second data in the second node and the target storage address for storing the calculation result.

[0061] In the disclosed embodiment, the first data is transmitted and the first primitive operation is indicated through the first message. This method makes the data communication of the distributed cluster more flexible. The various fields in the message, such as the operation code and the aggregate calculation type identifier, can be customized according to specific needs, so as to realize different types of computing operations and data processing, so that distributed training has higher configurability and adaptability. Secondly, this method of adding new primitive operations to the network adapter and instructing other nodes to perform corresponding primitive operations by sending messages can be used in different systems and environments if the network adapter and related software support the corresponding message format and operation protocol, providing a consistent interface and mechanism for data communication and computing in distributed clusters.

[0062] Continue to refer Figure 3A In block 320, in response to the first message, the second node may perform a first primitive operation on the data of the first node and the data of the second node through a network adapter to obtain an aggregate calculation result.

[0063] In some embodiments, after receiving the first message sent by the first node, the second node can parse the message to determine to perform the first primitive operation and the data sent by the first node that needs to be aggregated. In addition, the first primitive operation indicates to perform the aggregate calculation through the network adapter of the second node, thereby reducing multiple transmissions and processing steps of data between the network adapter and the GPU.

[0064] For example, please refer to Figure 4A , Figure 4A A schematic diagram of a process 400A for executing a first primitive operation in some embodiments of the present disclosure is shown in FIG. The process 400A may be implemented at the network adapter 111 of the node 110, and the following description will still be made by taking the network adapter of the second node as an example.

[0065] In block 410 , the network adapter of the second node obtains, in response to the first message, first data that needs to be aggregated and calculated in the first node and second data that needs to be aggregated and calculated in the second node.

[0066] In some embodiments, after receiving the first message, the network adapter determines to execute the first atomic operation through the operation code in the first message. The network adapter can directly obtain the data that needs to be aggregated and calculated sent by the first node from the data field in the first message. Then, the network adapter reads the local data that needs to be aggregated and calculated in the second node according to the memory address of the storage address of the second data that needs to be aggregated and calculated in the second node specified in the first message.

[0067] In the embodiment of the present disclosure, after receiving the first message, the network adapter can directly obtain the first data and the second data that need to be aggregated and calculated, avoiding the communication and data transmission between the CPU and the GPU, and reducing the delay. This method simplifies the process and operation of distributed communication and improves the training efficiency.

[0068] In some embodiments, after the second node completes receiving the first message, it can generate a completion signal to confirm that the second node has completed the data reception operation. For example, after completing the reception of the data, a completion signal will be received in the completion queue of the second node. This completion signal can inform the second node that new data has been written. The CPU of the second node can know the arrival of the data by polling the completion signal in its own completion queue, and perform corresponding processing according to the received data.

[0069] At block 420 , the network adapter of the second node performs an aggregate calculation on the first data and the second data.

[0070] In some embodiments, after obtaining the first data and the second data, the aggregation calculation can be performed directly on the network adapter of the second node. By performing the aggregation calculation directly on the network adapter, the computing load and communication process of the CPU and GPU are reduced, and the overall efficiency of distributed training is improved.

[0071] For example, please refer to Figure 4B , Figure 4B A schematic diagram of a process 400B for acquiring first data and second data in some embodiments of the present disclosure is shown in FIG. The process 400B may be implemented at the network adapter 111 of the node 110, and the following description will still be made by taking the network adapter of the second node as an example.

[0072] In block 411 , the network adapter of the second node parses the first data from the first message and parses the storage address of the second data in the second node.

[0073] In block 412, the network adapter of the second node obtains the second data from the storage address of the second data.

[0074] In distributed computing and communication, especially when processing machine learning training tasks, nodes need to frequently transmit data and perform calculations. In some embodiments, the network adapter of the second node parses the storage address of the first data and the second data in the second node from the first message, and obtains the second data from the address.

[0075] As an example, node 110-1 sends a message 113-1 (first message) to node 110-2. The message contains the first data that needs to be aggregated and calculated in node 110-1, as well as the information required for the aggregate calculation task, such as the operation type and the storage address of the second data that needs to be aggregated and calculated in node 110-2. The network adapter (NIC) 111-2 of node 110-2 receives the message and writes it into the cache. The network adapter 111-2 parses the message, first identifies the operation code in the message, and confirms that this is a message that requires the execution of the first primitive operation. Based on other fields in the message, the network adapter 111-2 parses the following key information: First data: directly extracted from the data field of the message. Storage address of the second data: extracted from the storage address field of the second data in the message, which contains the address information of the second data in the local memory of node 110-2. The network adapter 111-2 uses the parsed storage address to read the second data from the memory of node 110-2 and prepare to perform aggregate calculation with the first data.

[0076] In the embodiment of the present disclosure, the second data is directly read from the memory through the network adapter, thereby avoiding the transmission of data between the CPU and GPU of different nodes, reducing the delay, and reducing the data transmission steps and the consumption of hardware resources in this way.

[0077] For example, please refer to Figure 4C , Figure 4C A schematic diagram of a process 400C for performing aggregate computing in some embodiments of the present disclosure is shown in FIG. The process 400C may be implemented at the network adapter 111 of the node 110, and the following description will still be made by taking the network adapter of the second node as an example.

[0078] In block 421 , the network adapter determines a calculation type of the aggregate calculation by parsing the first message.

[0079] In some embodiments, the network adapter determines the calculation type of the aggregate calculation by parsing the aggregate calculation type identification field of the first message, such as addition, maximum value, minimum value, average value, logical AND, logical OR, XOR, etc.

[0080] At block 422, the network adapter sends the first data and the second data to the arithmetic logic unit.

[0081] At block 423, the network adapter performs a compute type aggregate calculation using the arithmetic logic unit.

[0082] In some embodiments, after determining the calculation type of the aggregate calculation, the network adapter may directly perform the corresponding type of aggregate calculation through its own arithmetic logic unit (ALU).

[0083] As an example, node 110-1 sends a message 113-1 (first message) to node 110-2. The message contains the first data that needs to be aggregated in node 110-1, and the information required for the aggregate calculation task, such as the operation type and the storage address of the second data that needs to be aggregated in node 110-2, and the calculation type of the aggregate calculation. The aggregate calculation type identification field determines the calculation type of the aggregate calculation, such as addition, maximum value, minimum value, average value, logical AND, logical OR, XOR, etc. Then the second data obtained from the memory is sent to the arithmetic logic unit (ALU) together with the first data. The ALU receives the first data and the second data, as well as the type information of the aggregate calculation. The ALU performs corresponding calculation operations on the two data according to the calculation type.

[0084] In the disclosed embodiment, the network adapter is not only responsible for data transmission, but also undertakes computing tasks. The aggregation computing tasks are completed on the network adapter, freeing up the computing resources of the GPU so that it can focus on other computing tasks. In addition, the aggregation computing is performed directly in the ALU through the network adapter, reducing the delay in data transmission between the CPU and the GPU.

[0085] Continue to refer Figure 3A , in block 330 , the result of the aggregate calculation is written back to the target storage address of the second node.

[0086] In some embodiments, the final result of the aggregate calculation performed on the second node needs to be written to a specified storage address so that related aggregate calculations and operations can be continued with other nodes in the distributed cluster.

[0087] In the disclosed embodiment, the calculation results are directly written back to the target storage address, avoiding additional data transmission and processing steps. The calculation and write-back operations are completed independently by the network adapter, thereby releasing key resources such as the CPU and GPU for use by other tasks. In this way, the context switching between the CPU and GPU for computing tasks and data transmission is reduced, and the overall efficiency of distributed training is improved.

[0088] For example, please refer to Figure 4D , Figure 4DA schematic diagram of the data calculation process of some embodiments of the present disclosure is shown in . Taking any node 110-i in the distributed cluster as an example, the network adapter of the node 110-i receives a message 113-j sent by other nodes. The message contains the data that needs to be aggregated and calculated in other nodes, as well as the information required for the aggregate calculation task. The data that needs to be aggregated and calculated in other nodes is saved in the cache Cache of the network adapter 111-i of the node 110-i, and then the data that needs to be aggregated and calculated in the node 110-i obtained from the memory 112-i is sent to the arithmetic logic unit ALU of the network adapter 111-i together with the data in the cache. Finally, the calculation result of the ALU is written back to the memory 112-i to complete the aggregate calculation operation, or the result is sent to the next node in the distributed cluster to continue the calculation.

[0089] In some embodiments, the calculation result obtained by performing aggregation calculation on the second node with the data of the first node needs to continue to perform aggregation calculation with other nodes to obtain the global parameters of the distributed cluster.

[0090] In some embodiments, a second message is sent from the second node to a third node in the distributed cluster via a network adapter, the second message indicating a second primitive operation, the second primitive operation indicating performing an aggregate calculation on a result of the aggregate calculation in the second node and data of the third node.

[0091] In distributed computing and communication scenarios, especially when performing distributed training in large-scale clusters, aggregate computing usually needs to be iterated among multiple nodes to ultimately obtain global parameters.

[0092] As an example, node 110-2 receives data from node 110-1 and performs preliminary aggregation calculation with local data to obtain preliminary calculation results. Since the preliminary calculation results only contain the aggregation results of part of the data, in order to obtain the global parameters of the entire distributed cluster, this result needs to be further aggregated with the data of other nodes. The network adapter 111-2 of node 110-2 can package the preliminary calculation results into message 113-2. Message 113-2 indicates the second primitive operation that needs to be performed, and the operation indicates that the preliminary calculation results in node 110-2 and the data in node 110-3 are aggregated through the network adapter 111-3 of node 110-3. After receiving message 113-2, the network adapter 111-3 of node 110-3 parses the message content and obtains the preliminary calculation results that need to be aggregated and the data in node 110-3. According to the instructions in message 113-2, the network adapter 111-3 performs aggregation calculations, calculates the preliminary calculation results of the node with the data in node 110-3, and obtains a more complete aggregation result. Furthermore, after the aggregate calculation is completed, the node 110-3 may also send the calculation result to the next node through the message 113-3 to continue the aggregate calculation. This iteration is repeated until the data of each node in the distributed calculation is aggregated and finally the global parameter is obtained.

[0093] In some embodiments, the first primitive operation is configured based on remote direct data access technology. Remote Direct Memory Access (RDMA) technology is a technology that enables computer storage to directly read and write remote memory without the intervention of the operating system. RDMA primitives include RDMA_READ, RDMA_WRITE, and SEND / RECEIVE. These primitives are used to perform basic data transfer operations and are usually implemented at the network adapter layer.

[0094] The first primitive operation is a new operation implemented by extending the RDMA communication primitive. Its main feature is that it combines data transmission and calculation, and directly performs data aggregation calculation through the network adapter. In order to configure this type of primitive operation in a distributed cluster, the specific operation performed by the network adapter can be defined through the RDMA primitive and the relevant protocol fields can be added to configure the first primitive operation.

[0095] In the disclosed embodiment, by combining data transmission and calculation, the network adapter is configured to directly perform the first primitive operation of data aggregation calculation, thereby providing an efficient and low-latency data calculation solution for distributed training scenarios, which is particularly suitable for distributed systems that need to perform frequent data aggregation calculations.

[0096] Figure 5A schematic structural block diagram of an apparatus 500 for data calculation according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in a node 110. Each module / component in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0097] As shown in the figure, the device 500 includes a receiving module 510, which is configured to receive a first message sent by a first node in a distributed cluster at a second node in the distributed cluster via a network adapter of the second node, the first message indicating a first primitive operation, and the first primitive operation indicating the execution of an aggregate calculation on the data of the first node and the data of the second node; a computing module 520, which is configured to execute the first primitive operation on the data of the first node and the data of the second node through the network adapter in response to the first message to obtain a result of the aggregate calculation; and a write-back module 530, which is configured to write the result of the aggregate calculation back to the target storage address of the second node.

[0098] In some embodiments, the device 500 also includes a sending module 540, which is configured to send a second message from the second node to a third node in the distributed cluster via a network adapter, the second message indicating a second primitive operation, and the second primitive operation indicating the performance of an aggregate calculation on the result of the aggregate calculation in the second node and the data of the third node.

[0099] In some embodiments, the first message includes at least one of the following: first data that needs aggregate calculation in the first node, an operation code, the operation code indicates a first primitive operation, an aggregate calculation type identifier, the aggregate calculation type identifier indicates the calculation type of the aggregate calculation of the first primitive operation, the storage address of the second data that needs aggregate calculation in the second node, and the target storage address.

[0100] In some embodiments, the computing module 520 is further configured to, in response to the first message, obtain first data that requires aggregate computing in the first node and second data that requires aggregate computing in the second node; and perform aggregate computing on the first data and the second data.

[0101] In some embodiments, the computing module 520 is further configured to parse the first data from the first message and parse the storage address of the second data in the second node; and obtain the second data from the storage address of the second data.

[0102] In some embodiments, the calculation module 520 is further configured to determine the calculation type of the aggregate calculation by parsing the first message; send the first data and the second data to the arithmetic logic unit of the network adapter; and use the arithmetic logic unit to perform the calculation type of the aggregate calculation.

[0103] In some embodiments, the first primitive operation is configured based on remote direct data access technology.

[0104] Figure 6 6 shows a block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 6 The electronic device 600 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to implement Figure 2A Node 110 or Figure 5 Device 500.

[0105] like Figure 6 As shown, the electronic device 600 is in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 620. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capabilities of the electronic device 600.

[0106] The electronic device 600 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 600.

[0107] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 6As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0108] The communication unit 640 implements communication with other computing devices through a communication medium. Additionally, the functions of the components of the electronic device 600 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 600 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0109] The input device 650 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 660 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 600, or communicate with any device that allows the electronic device 600 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0110] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0111] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0112] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0113] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0114] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0115] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A data calculation method, comprising: At a second node in the distributed cluster, receiving, via a network adapter of the second node, a first message sent by a first node in the distributed cluster, the first message indicating a first primitive operation, the first primitive operation indicating performing aggregate calculation on data of the first node and data of the second node; In response to the first message, performing the first primitive operation on the data of the first node and the data of the second node through the network adapter to obtain the result of the aggregate calculation; as well as The result of the aggregate calculation is written back to the target storage address of the second node.

2. The method according to claim 1, further comprising: A second message is sent from the second node to a third node in the distributed cluster via the network adapter, the second message indicating a second primitive operation, the second primitive operation indicating performing an aggregate calculation on a result of the aggregate calculation in the second node and data of the third node.

3. The method according to claim 1, wherein the first message includes at least one of the following: The first data in the first node that needs to be aggregated and calculated, an operation code indicating the first primitive operation, an aggregate calculation type identifier, wherein the aggregate calculation type identifier indicates a calculation type of the aggregate calculation of the first primitive operation, The storage address of the second data that needs to be aggregated and calculated in the second node in the second node, The target storage address.

4. The method of claim 1 , wherein performing the first primitive operation comprises: In response to the first message, acquiring first data that needs to be aggregated and calculated in the first node and second data that needs to be aggregated and calculated in the second node; as well as An aggregate calculation is performed on the first data and the second data.

5. The method according to claim 4, wherein acquiring the first data and the second data comprises: Parse the first data from the first message and parse the storage address of the second data in the second node; as well as The second data is acquired from the storage address of the second data.

6. The method according to claim 4, wherein performing aggregate calculation on the first data and the second data comprises: Determine the calculation type of the aggregate calculation by parsing the first message; sending the first data and the second data to an arithmetic logic unit of the network adapter; as well as An aggregate calculation of the calculation type is performed using the arithmetic logic unit.

7. The method of claim 1, wherein the first primitive operation is configured based on remote direct data access technology.

8. A data computing device, comprising: A receiving module is configured to receive, at a second node in the distributed cluster, via a network adapter of the second node, a first message sent by a first node in the distributed cluster, wherein the first message indicates a first primitive operation, and the first primitive operation indicates performing an aggregate calculation on data of the first node and data of the second node; a computing module configured to, in response to the first message, perform the first primitive operation on the data of the first node and the data of the second node through the network adapter to obtain a result of the aggregate computing; as well as A write-back module is configured to write the result of the aggregate calculation back to the target storage address of the second node.

9. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 7 when executed by the at least one processing unit.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.