Data reduction method and apparatus, and device and computer-readable storage medium

Abstract

The present application belongs to the technical field of communications. Disclosed are a data reduction method and apparatus, and a device and a computer-readable storage medium. The method is applied to at least one root node in a multi-dimensional direct-connect topology, wherein nodes in the multi-dimensional direct-connect topology are used for model training. The method comprises: receiving first data, wherein the first data is part of transmitted data, which is data transmitted between nodes in a model training process; obtaining second data, wherein the second data is part of first generated data, which is data generated by a root node in the model training process; performing reduction on the first data and the second data to obtain third data, wherein the third data is used by nodes in a multi-dimensional direct-connect topology to execute subsequent model training; and sending the third data to a first node in the multi-dimensional direct-connect topology, wherein the first node is a node in the multi-dimensional direct-connect topology other than the root node. The method can improve the communication efficiency in a data reduction process.

Description

Data reduction method, device, equipment and computer readable storage medium

[0001] The present application claims priority to the Chinese patent application No. 202410620153.X, filed on May 17, 2024, and entitled "Data reduction method, device, equipment and computer readable storage medium", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of communication technology, in particular to a data reduction method, device, equipment and computer readable storage medium. BACKGROUND

[0003] In some scenarios in the field of communication technology, data of multiple nodes in the same network will be reduced. The reduction refers to aggregating distributed data on multiple nodes into a global calculation result. By reducing the data on multiple nodes in the network and distributing the reduced result to each node in the network, each node in the network can perform a global data processing task in parallel based on the same reduced result, thereby realizing the unified coordination and management of the network. SUMMARY

[0004] The present application provides a data reduction method, device, equipment and computer readable storage medium to improve the efficiency of data reduction. The technical solutions are as follows:

[0005] In a first aspect, a data reduction method is provided. The method is applied to at least one root node in a multi-dimensional direct connection topology. The nodes in the multi-dimensional direct connection topology are used for model training. The method comprises: receiving first data, the first data being part of transmission data, the transmission data being data transmitted between nodes during model training; obtaining second data, the second data being part of first generated data, the first generated data being data generated by the root node during model training; reducing the first data and the second data to obtain third data, the third data being used for the nodes in the multi-dimensional direct connection topology to perform subsequent model training; and sending the third data to a first node in the multi-dimensional direct connection topology, the first node being a node in the multi-dimensional direct connection topology other than the root node.

[0006] In the present application, the first data received by the root node is part of the transmission data, and the second data obtained is part of the first generated data. The volume of the first data and the second data is small, the efficiency of reducing the first data and the second data is high, and the required computing resources are less, thereby making the efficiency of data reduction higher. The root node sends the third data to the nodes in the multi-dimensional direct connection topology, so that the nodes in the multi-dimensional direct connection topology can cooperatively perform subsequent model training based on the same third data, thereby ensuring the accuracy of model training.

[0007] In a possible implementation, the first data is data corresponding to a first path in the multi-dimensional direct connection topology, and the first node is a node on the first path; before receiving the first data, the method further includes: receiving first path information sent by the first node, the first path information being used by the root node to determine the first path. By receiving the first path information sent by the first node, the root node can determine the first path in the multi-dimensional direct connection topology, so as to determine that the first data is data corresponding to the first path, obtain second data corresponding to the first path, and enable the first data and the second data to be accurately reduced.

[0008] In a possible implementation, before receiving the first data, the method further includes: obtaining second path information, the second path information indicating a path in the multi-dimensional direct connection topology determined by the root node; and broadcasting a first packet to the first node, the first packet being used to indicate information of the path in the multi-dimensional direct connection topology determined by the root node.

[0009] The root node broadcasts the first packet to the first node, so that the first node can be aware of the path in the multi-dimensional direct connection topology determined by the root node, and ensure that the first node and the root node confirm the path in the multi-dimensional direct connection topology consistently.

[0010] In a possible implementation, before receiving the first data, the method further includes: receiving first path information sent by the first node, the first path information being used by the root node to determine the first path; obtaining second path information, the second path information indicating a path in the multi-dimensional direct connection topology determined by the root node; and broadcasting a first packet to the first node in a case where the first path information does not conflict with the second path information, the first packet being used to indicate the path in the multi-dimensional direct connection topology determined by the root node.

[0011] The first path information does not conflict with the second path information, which means that the first path indicated by the first path information and the path in the multi-dimensional direct connection topology indicated by the second path information do not conflict, and thus the first path determined by the first node and the path in the multi-dimensional direct connection topology determined by the root node are both valid paths. Therefore, the root node can send the first packet to the first node, so as to ensure that the first node and the root node confirm the path in the multi-dimensional direct connection topology consistently.

[0012] In a possible implementation, the first packet includes at least one of the following contents: an identifier of the first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of the root node; an identifier of a node sending the first packet; an identifier of a second node, the second node being a node subsequent to the root node on the first path; and a packet type of the first packet. Through the various information in the first packet, the root node and the path determined by the root node in the multi-dimensional direct connection topology can be accurately indicated, so as to ensure that the first node and the root node confirm the path in the multi-dimensional direct connection topology consistently.

[0013] In a possible implementation, the first data is carried by a second packet, the second packet further includes an identifier of a path corresponding to the first data and an identifier of the first data, the identifier of the path corresponding to the first data is used to determine the first generated data, and the identifier of the first data is used to determine the second data in the first generated data. Through the identifier of the path corresponding to the first data and the identifier of the first data, the second data corresponding to the first data can be accurately determined, so that the corresponding first data and second data are reduced to obtain an accurate reduction result.

[0014] In a possible implementation, the second data is obtained by splitting the first generated data. Through splitting the first generated data, the second data with a smaller volume can be obtained, so that the efficiency of data reduction is improved.

[0015] In a possible implementation, the first data and the second data are reduced to obtain third data, including: reducing the first data and the second data according to a first reduction order to obtain the third data, and the first reduction order is determined based on a data transmission order on a path corresponding to the first data and the second data. According to the specified reduction order, the accuracy of the reduction result can be ensured, and the reproducibility of the reduction result can be ensured.

[0016] In a second aspect, a data reduction method is provided, and the method is applied to a third node. The third node is a node in a multi-dimensional direct connection topology except a root node. The nodes in the multi-dimensional direct connection topology are used for model training. The method includes: receiving fourth data, the fourth data being part of transmission data, the transmission data being data transmitted between nodes in a model training process; obtaining fifth data, the fifth data being part of second generated data, the second generated data being data generated by the third node in the model training process; reducing the fourth data and the fifth data to obtain sixth data, the sixth data being used for the root node to obtain third data, the third data being used for the nodes in the multi-dimensional direct connection topology to perform subsequent model training.

[0017] In the present application, the fourth data received by the third node is part of the transmission data, and the fifth data obtained is part of the second generated data. The fourth data and the fifth data have a smaller volume, and the reduction of the fourth data and the fifth data has a higher efficiency and requires less computing resources, so that the efficiency of data reduction is higher.

[0018] In a possible implementation, the fourth data is data corresponding to a first path in the multi-dimensional direct connection topology, and the third node is a node on the first path; before receiving the fourth data, the method further includes: sending first path information to the root node, the first path information being used by the root node to determine the first path; and sending the first path information to the root node enables the root node to determine the first path in the multi-dimensional direct connection topology, so that the root node and the third node are consistent in confirming the first path.

[0019] In a possible implementation, before receiving the fourth data, the method further includes: receiving a first message sent by the root node, the first message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the root node; and broadcasting a third message to nodes in the multi-dimensional direct connection topology, the third message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the third node. By receiving the first message, the third node can determine the path in the multi-dimensional direct connection topology determined by the root node, so that the third node and the root node are consistent in confirming the path in the multi-dimensional direct connection topology.

[0020] In a possible implementation, the third message includes at least one of the following: an identifier of the first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of a root node on the first path; an identifier of the third node; an identifier of a fourth node, the fourth node being a node after the third node on the first path; an identifier of a fifth node, the fifth node being a node before the third node on the first path; and a message type of the third message. By using the information in the third message, the third node and the root node can accurately indicate information of a path in the multi-dimensional direct connection topology determined by the third node and the root node, so that the third node and other nodes in the multi-dimensional direct connection topology are consistent in confirming the path in the multi-dimensional direct connection topology.

[0021] In a possible implementation, the fourth data is carried in a fourth message, and the fourth message further includes an identifier of a path corresponding to the fourth data and an identifier of the fourth data, the identifier of the path corresponding to the fourth data being used to determine the second generated data, and the identifier of the fourth data being used to determine the fifth data in the second generated data. By using the identifier of the path corresponding to the fourth data and the identifier of the fourth data, the fifth data corresponding to the fourth data can be accurately determined, so that the corresponding fourth data and fifth data are reduced to obtain an accurate reduction result.

[0022] In a possible implementation, the fifth data is obtained by: splitting the second generated data to obtain the fifth data. By splitting the second generated data, the fifth data with a smaller volume can be obtained, so that the efficiency of data reduction is improved.

[0023] In a possible implementation, the fourth data and the fifth data are reduced to obtain the sixth data, including: the fourth data and the fifth data are reduced according to a second reduction order to obtain the sixth data, and the second reduction order is determined based on a data transmission sequence on a path corresponding to the fourth data and the fifth data. The data reduction according to the specified reduction order can ensure the accuracy of the reduction result and guarantee the reproducibility of the reduction result.

[0024] In a possible implementation, the fourth data and the fifth data are reduced to obtain the sixth data, including: in a case where the computing resource of the third node meets a resource condition, the fourth data and the fifth data are reduced to obtain the sixth data, and the resource condition is used to indicate that the computing resource of the third node is greater than a resource required for reducing the fourth data and the fifth data. In the case where the computing resource of the third node meets the resource condition, the fourth data and the fifth data are reduced, which can avoid reduction failure and improve the success rate of reduction.

[0025] In a possible implementation, after the fourth data is received, the method further includes: in a case where the computing resource of the third node does not meet a resource condition, the fourth data and the fifth data are sent to a sixth node, and the resource condition is used to indicate that the computing resource of the third node is greater than a resource required for reducing the fourth data and the fifth data, and the sixth node is a previous node of the third node on a path corresponding to the fourth data. In the case where the computing resource of the third node does not meet the resource condition, the fourth data and the fifth data are not reduced, and the fourth data and the fifth data are directly sent to the sixth node, which can avoid reduction failure and improve the success rate of reduction.

[0026] In a third aspect, a data reduction method is provided, and the method is applied to a seventh node. The seventh node is a node in a multi-dimensional direct connection topology except a root node. The nodes in the multi-dimensional direct connection topology are used for model training. The method includes: obtaining seventh data, the seventh data being part of third generated data, and the third generated data being data generated by the seventh node in a model training process; and sending the seventh data to the root node, the seventh data being used by the root node to obtain third data, and the third data being used by the nodes in the multi-dimensional direct connection topology to perform subsequent model training.

[0027] In this application, the seventh data obtained by the seventh node is part of the third generated data, the volume of the seventh data is small, the efficiency of reducing the seventh data is high, and the required computing resource is less, so that the efficiency of data reduction is high.

[0028] In a possible implementation, before obtaining the seventh data, the method further includes: sending first path information to the root node, the first path information being used for the root node to determine the first path. The first path information is sent to the root node, so that the root node can determine the first path in the multi-dimensional direct connection topology, and the root node and the seventh node are consistent in confirming the first path.

[0029] In a possible implementation, before obtaining the seventh data, the method further includes: receiving a first message sent by the root node, the first message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the root node; and broadcasting a fifth message to nodes in the multi-dimensional direct connection topology, the fifth message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the seventh node. The first message is received, so that the seventh node can determine the path in the multi-dimensional direct connection topology determined by the root node, and the seventh node and the root node are consistent in confirming the path in the multi-dimensional direct connection topology.

[0030] In a possible implementation, the fifth message includes at least one of the following: an identifier of the first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of the root node on the first path; an identifier of the seventh node; an identifier of an eighth node, the eighth node being a previous node of the seventh node on the first path; and a message type of the fifth message. The information of the path in the multi-dimensional direct connection topology determined by the root node and the seventh node can be accurately indicated by the information in the fifth message, so that the seventh node and other nodes in the multi-dimensional direct connection topology are consistent in confirming the path in the multi-dimensional direct connection topology.

[0031] In a possible implementation, the seventh data is obtained by: splitting the third generated data.

[0032] In any possible implementation of the method provided in the first aspect to the third aspect, the method further includes: obtaining identifiers of nodes in the multi-dimensional direct connection topology, the identifier of any node indicating a dimension and a position of the any node in the multi-dimensional direct connection topology; and determining the path and the at least one root node in the multi-dimensional direct connection topology according to the identifiers of the nodes in the multi-dimensional direct connection topology.

[0033] Since the identifier of the node in the multi-dimensional direct connection topology can indicate the dimension and the position of the node in the multi-dimensional direct connection topology, the node in the multi-dimensional direct connection topology can determine the at least one root node and the path in the multi-dimensional direct connection topology according to the identifier of the node in the multi-dimensional direct connection topology.

[0034] In any possible implementation of the method provided in the first aspect to the third aspect, the at least one root node is determined based on load information of the nodes in the multi-dimensional direct connection topology. According to the load information of the nodes in the multi-dimensional direct connection topology, the load capacity of the nodes in the multi-dimensional direct connection topology can be determined, so that the root node with stronger load capacity can be accurately determined.

[0035] In a fourth aspect, a data reduction apparatus is provided. The apparatus is applied to at least one root node in a multi-dimensional direct connection topology, and the nodes in the multi-dimensional direct connection topology are used for model training. The apparatus comprises: a first transceiver module, configured to receive first data, the first data being part of transmission data, and the transmission data being data transmitted between the nodes in the model training process; a first processing module, configured to obtain second data, the second data being part of first generated data, and the first generated data being data generated by the root node in the model training process; reduce the first data and the second data to obtain third data, the third data being used for subsequent model training of the nodes in the multi-dimensional direct connection topology; and the first transceiver module is configured to send the third data to a first node in the multi-dimensional direct connection topology, the first node being a node in the multi-dimensional direct connection topology except the root node.

[0036] In a possible implementation, the first data is data corresponding to a first path in the multi-dimensional direct connection topology, and the first node is a node on the first path; the first transceiver module is further configured to receive first path information sent by the first node, and the first path information is used for the root node to determine the first path.

[0037] In a possible implementation, the first processing module is further configured to obtain second path information, the second path information indicating a path in the multi-dimensional direct connection topology determined by the root node; and the first transceiver module is further configured to broadcast a first packet to the first node, the first packet being used to indicate information of the path in the multi-dimensional direct connection topology determined by the root node.

[0038] In a possible implementation, the first packet comprises at least one of the following contents: an identifier of the first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of the root node; an identifier of a node sending the first packet; an identifier of a second node, the second node being a node after the root node on the first path; and a type of the first packet.

[0039] In a possible implementation, the first data is carried by a second packet, and the second packet further comprises an identifier of a path corresponding to the first data and an identifier of the first data, the identifier of the path corresponding to the first data being used to determine the first generated data, and the identifier of the first data being used to determine the second data in the first generated data.

[0040] In a possible implementation, the first processing module is configured to split the first generated data to obtain the second data.

[0041] In a possible implementation, the first processing module is configured to perform reduction on the first data and the second data according to a first reduction order to obtain third data, and the first reduction order is determined based on a data transmission order on a path corresponding to the first data and the second data.

[0042] In a fifth aspect, a data reduction apparatus is provided. The apparatus is applied to a third node, and the third node is a node in a multi-dimensional direct connection topology except a root node. The nodes in the multi-dimensional direct connection topology are configured to perform model training. The apparatus comprises: a second transceiver configured to receive fourth data, the fourth data being part of transmission data, and the transmission data being data transmitted between nodes in a model training process; a second processing module configured to obtain fifth data, the fifth data being part of second generated data, and the second generated data being data generated by the third node in the model training process; perform reduction on the fourth data and the fifth data to obtain sixth data, and the sixth data being used by the root node to obtain third data, and the third data being used by the nodes in the multi-dimensional direct connection topology to perform subsequent model training; and the second transceiver is configured to send the sixth data to the root node.

[0043] In a possible implementation, the fourth data is data corresponding to a first path in the multi-dimensional direct connection topology, and the third node is a node on the first path; and the second transceiver is further configured to send first path information to the root node, and the first path information being used by the root node to determine the first path.

[0044] In a possible implementation, the second transceiver is further configured to receive a first message sent by the root node, and the first message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the root node; and broadcast a third message to the nodes in the multi-dimensional direct connection topology, and the third message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the third node.

[0045] In a possible implementation, the third message comprises at least one of the following contents: an identifier of the first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of a root node on the first path; an identifier of the third node; an identifier of a fourth node, the fourth node being a node after the third node on the first path; an identifier of a fifth node, the fifth node being a node before the third node on the first path; and a message type of the third message.

[0046] In a possible implementation, the fourth data is carried by a fourth message, and the fourth message further comprises an identifier of a path corresponding to the fourth data and an identifier of the fourth data, and the identifier of the path corresponding to the fourth data being used to determine the second generated data, and the identifier of the fourth data being used to determine the fifth data in the second generated data.

[0047] In a possible implementation, the second processing module is configured to split the second generated data to obtain fifth data.

[0048] In a possible implementation, the second processing module is configured to perform reduction on the fourth data and the fifth data according to a second reduction order to obtain sixth data, the second reduction order being determined based on data transmission orders on paths corresponding to the fourth data and the fifth data.

[0049] In a possible implementation, the second processing module is configured to perform reduction on the fourth data and the fifth data to obtain sixth data in a case where the computing resource of the third node meets a resource condition, the resource condition being used to indicate that the computing resource of the third node is greater than a resource required for reduction on the fourth data and the fifth data.

[0050] In a possible implementation, the second transceiver is configured to send the fourth data and the fifth data to a sixth node in a case where the computing resource of the third node does not meet a resource condition, the resource condition being used to indicate that the computing resource of the third node is greater than a resource required for reduction on the fourth data and the fifth data, the sixth node being a previous node of the third node on a path corresponding to the fourth data.

[0051] In a sixth aspect, a data reduction apparatus is provided, and the apparatus is applied to a seventh node, the seventh node being a node in a multi-dimensional direct connection topology except a root node, the nodes in the multi-dimensional direct connection topology being used to perform model training, and the apparatus comprises: a third processing module configured to obtain seventh data, the seventh data being part of third generated data, the third generated data being data generated by the seventh node in a model training process; and a third transceiver configured to send the seventh data to the root node, the seventh data being used by the root node to obtain third data, the third data being used by the nodes in the multi-dimensional direct connection topology to perform subsequent model training.

[0052] In a possible implementation, the seventh node is a node on a first path in the multi-dimensional direct connection topology; and the third transceiver is further configured to send first path information to the root node, the first path information being used by the root node to determine the first path.

[0053] In a possible implementation, the third transceiver is further configured to receive a first message sent by the root node, the first message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the root node; and broadcast a fifth message to the nodes in the multi-dimensional direct connection topology, the fifth message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the seventh node.

[0054] In a possible implementation, the fifth message comprises at least one of the following: an identifier of the first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of the root node on the first path; an identifier of the seventh node; an identifier of the eighth node, the eighth node being a previous node of the seventh node on the first path; a message type of the fifth message.

[0055] In a possible implementation, the third processing module is configured to split the third generated data to obtain the seventh data.

[0056] In a possible implementation, the first processing module, the second processing module, or the third processing module is further configured to obtain identifiers of nodes in the multi-dimensional direct connection topology, the identifier of any node indicating a dimension and a position of the any node in the multi-dimensional direct connection topology; and determine the paths and the at least one root node in the multi-dimensional direct connection topology according to the identifiers of the nodes in the multi-dimensional direct connection topology.

[0057] In a possible implementation, the at least one root node is determined based on load information of the nodes in the multi-dimensional direct connection topology.

[0058] In a seventh aspect, a data reduction system is provided, the data reduction system comprising a root node, a third node, and a seventh node, the root node being configured to perform the data reduction method in the first aspect and any possible implementation thereof, the third node being configured to perform the data reduction method in the second aspect and any possible implementation thereof, and the seventh node being configured to perform the data reduction method in the third aspect and any possible implementation thereof.

[0059] In an eighth aspect, a data reduction device is provided, the device comprising a processor and a memory, the memory being coupled to the processor; and the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to enable the data reduction device to implement the data reduction method in the first aspect and any possible implementation thereof, or implement the data reduction method in the second aspect and any possible implementation thereof, or implement the data reduction method in the third aspect and any possible implementation thereof.

[0060] In a ninth aspect, a computer program (product) is provided, the computer program (product) comprising computer program code which, when executed by a computer, causes the computer to perform the method in the above aspects.

[0061] In a tenth aspect, a computer-readable storage medium is provided, the computer-readable storage medium storing a program or instructions, the method in the above aspects being performed when the program or instructions are executed on a computer.

[0062] In an eleventh aspect, a chip is provided, comprising a processor configured to invoke and run instructions stored in a memory, so that a computer installed with the chip executes the method in any of the aspects above.

[0063] In a twelfth aspect, another chip is provided, comprising an input interface, an output interface, a processor and a memory, the input interface, the output interface, the processor and the memory are connected through internal connection paths, the processor is configured to execute a code in the memory, when the code is executed, a computer installed with the chip executes the method in any of the aspects above.

[0064] It should be understood that the technical solutions of the fourth aspect to the twelfth aspect of the present application and the corresponding possible implementation manners have the beneficial effects as described above for the first aspect, the second aspect and the third aspect and the corresponding possible implementation manners, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0065] FIG. 1 is a schematic diagram of a multi-dimensional direct connection topology provided by an embodiment of the present application;

[0066] FIG. 2 is a flowchart of a data reduction method executed by a seventh node provided by an embodiment of the present application;

[0067] FIG. 3 is a 4*4 two-dimensional torus type topology provided by an embodiment of the present application;

[0068] FIG. 4 is a schematic diagram of a multi-dimensional direct connection topology provided by an embodiment of the present application;

[0069] FIG. 5 is a schematic diagram of a path tree in a multi-dimensional direct connection topology provided by an embodiment of the present application;

[0070] FIG. 6 is a schematic diagram of a path tree in another multi-dimensional direct connection topology provided by an embodiment of the present application;

[0071] FIG. 7 is a flowchart of a data reduction method executed by a third node provided by an embodiment of the present application;

[0072] FIG. 8 is a schematic diagram of a fourth packet provided by an embodiment of the present application;

[0073] FIG. 9 is a schematic diagram of another fourth packet provided by an embodiment of the present application;

[0074] FIG. 10 is a schematic diagram of data reduction provided by an embodiment of the present application;

[0075] FIG. 11 is a schematic diagram of another data reduction provided by an embodiment of the present application;

[0076] FIG. 12 is a flowchart of a data reduction method executed by a root node provided by an embodiment of the present application;

[0077] FIG. 13 is a schematic diagram of a complete data reduction process according to an embodiment of the present application;

[0078] FIG. 14 is a schematic diagram of a node according to an embodiment of the present application;

[0079] FIG. 15 is a schematic diagram of a time delay according to an embodiment of the present application;

[0080] FIG. 16 is a diagram of an implementation scenario according to the related art;

[0081] FIG. 17 is a flowchart of an operation of performing data reduction according to the related art;

[0082] FIG. 18 is a schematic diagram of a structure of a data reduction apparatus according to an embodiment of the present application;

[0083] FIG. 19 is a schematic diagram of a structure of another data reduction apparatus according to an embodiment of the present application;

[0084] FIG. 20 is a schematic diagram of a structure of yet another data reduction apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0085] The terms used in the embodiments section of the present application are used only to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0086] All-reduce is a collective communication operation, which aims to collect and calculate data distributed on different nodes to obtain a global calculation result, and enable each node to obtain the global calculation result. The all-reduce operation can effectively realize the collection of distributed data and the synchronization of the global calculation result in large-scale parallel computing, and provides basic support for coordinating and managing multiple nodes to perform global calculation tasks.

[0087] For example, an all-reduce operation can be used in the process of model training and model inference of a large model (LM). The large model refers to a deep neural network (DNN) model with a large number of parameters (tens of billions or even hundreds of billions) and a complex computing structure. The large model has been widely applied in various fields, covering natural language processing (NLP), computer vision (CV), and multimodal fields. The process of model training and model inference of the large model requires large-scale data sets and a large amount of computing resources and communication resources, involves large-scale parallel computing, and the large model needs to be collaboratively trained by multiple nodes. Therefore, through all-reduce, the multiple nodes can obtain the global computing result of the large model, so that the multiple nodes can collaboratively train the large model based on the same global computing result.

[0088] The direct connection topology is one of the mainstream networking methods of an artificial intelligence (AI) computing cluster for model training and model inference of a large model. Each node in the direct connection topology is equipped with a device with a forwarding function. The device with the forwarding function is both the data generation and sink end and the data transfer station. The direct connection topology can provide efficient data communication and computing capability, support large-scale data transmission and parallel computing, and have certain fault tolerance capability. However, when data reduction is performed in the direct connection topology, there is a competition phenomenon for memory resources such as high bandwidth memory (HBM) and double data rate synchronous dynamic random-access memory (DDR SDRAM) and bandwidth of the nodes, resulting in low communication efficiency when data reduction is performed in the direct connection topology.

[0089] Embodiments of the present application provide a data reduction method, which can improve the communication efficiency and bandwidth utilization of data reduction in the direct connection topology. The implementation scenario of the data reduction method provided by the embodiments of the present application is a scenario including a multi-dimensional direct connection topology. The multi-dimensional direct connection topology includes multiple nodes, and the multiple nodes can be nodes for model training. The multiple nodes are distributed in multiple dimensions, and the distribution of the multiple nodes in the multiple dimensions can be uniform distribution or non-uniform distribution. Each node is connected or deployed with a device with a forwarding function, and each node realizes data reception and transmission through the device with the forwarding function. In the embodiments of the present application, each node can also perform data reduction through the device with the forwarding function.

[0090] The node can also be referred to as a computing node, and can include one or more combinations of a processor, a storage device, a network interface, an input / output processing unit, and an accelerator. Among them, the processor is the core of the node, responsible for performing computing tasks and controlling the operation of the node. The processor can be, for example, a central processing unit (CPU), a graphics processing unit (GPU), or a tensor processing unit (TPU), etc. The storage device is used to store data and program instructions, and provides the function of reading and writing data of the node. The storage device can be, for example, a hard disk, a memory card, an optical disk, a virtual memory, an HBM, or a DDR, etc. The network interface is used for communication and data transmission between nodes. The network interface can be, for example, a fiber interface, a serial interface, or a multiplexing interface, etc. The input / output processing unit is responsible for the data input / output operation of the node, and manages the data interaction between the node and the node. The input / output processing unit can be, for example, an input / output die (I / O Die) or a data processing unit (DPU), etc. The accelerator is used to accelerate specific types of computing tasks and improve computing efficiency. The accelerator can be, for example, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), etc. Multiple nodes can provide parallel computing, storage and data management, network communication, etc. for the job process of the implementation scenario.

[0091] The implementation scenario can be various large-scale parallel computing scenarios that require efficient parallel data reduction, such as devices or device clusters deployed with large models, distributed computing systems, edge computing systems, or virtual computing systems in cloud computing platforms, etc. The types of multiple nodes in different implementation scenarios are different. For example, if the implementation scenario is a device deployed with a large model, such as a computer deployed with a large model, the multiple nodes can be one or more combinations of chips, processors, registers, or network cards in the device deployed with a large model; if the implementation scenario is a device cluster deployed with a large model, the multiple nodes can be multiple devices in the device cluster; if the implementation scenario is a distributed computing system, the multiple nodes can be multiple computer nodes in the distributed computing system; if the implementation scenario is an edge computing system, the multiple nodes can be multiple edge computing nodes in the edge computing system; and if the implementation scenario is a virtual computing system in a cloud computing platform, the multiple nodes can be multiple virtual computing nodes in the virtual computing system.

[0092] Optionally, in the embodiments of the present application, the types of the plurality of nodes can be the same or different. For example, when the implementation scenario is a device cluster deployed with a large model, node 1 in the plurality of nodes can be a chip on device 1 in the device cluster, and node 2 can be a processor on device 2 in the device cluster.

[0093] Regardless of the implementation scenario, by implementing the data reduction method provided in the embodiments of the present application, the communication efficiency during data reduction can be improved, and the response efficiency and resource utilization of each node can be improved.

[0094] Referring to FIG. 1, a schematic diagram of a multi-dimensional direct connection topology provided by an embodiment of the present application is shown. The multi-dimensional direct connection topology shown in FIG. 1 is a two-dimensional direct connection topology, including 16 nodes represented by nodes 0 to 15, and the 16 nodes form a plane with a specification of 4*4.

[0095] It should be noted that FIG. 1 is only an example of an implementation scenario, and the embodiments of the present application do not limit the dimensions of the multi-dimensional direct connection topology and the number of nodes in each dimension. Moreover, the embodiments of the present application do not limit the type of multi-dimensional direct connection topology and the connection relationship between nodes. For example, in a mesh type multi-dimensional direct connection topology, each node is connected only to adjacent nodes. In a torus type multi-dimensional direct connection topology, each node is connected to adjacent nodes, and the edge nodes in any dimension are connected to another edge node in the same dimension. In a hyperX (a type of direct connection topology) type multi-dimensional direct connection topology, any node is connected to the remaining nodes in the same dimension of the multi-dimensional direct connection topology. In a ring type multi-dimensional direct connection topology, any node is connected to adjacent nodes, and a plurality of nodes are connected in a specified direction to form a ring.

[0096] In the embodiments of the present application, the plurality of nodes in the multi-dimensional direct connection topology includes a root node and leaf nodes. The root node is a destination node for the aggregation of data of each node in the multi-dimensional direct connection topology in a full reduction process. The leaf nodes are nodes other than the root node. The data on the leaf nodes are all transmitted to the root node, and the root node performs full reduction on the data of the nodes in the multi-dimensional direct connection topology.

[0097] Optionally, the root node in the multi-dimensional direct connection topology can be one or more. In different full reduction rounds, the root node in the multi-dimensional direct connection topology can be different. For example, in a device deployed with a large model, different rounds of model training or model inference performed by a plurality of nodes correspond to different computing tasks, and different computing tasks have different reduction requirements. Therefore, in different full reduction rounds, the root node in the multi-dimensional direct connection topology can be different. The root node in the multi-dimensional direct connection topology can be manually specified and statically configured, or can be determined by election by the plurality of nodes in the multi-dimensional direct connection topology.

[0098] Exemplarily, the at least one root node in the multi-dimensional direct connection topology can be determined based on the load information of the nodes in the multi-dimensional direct connection topology. In the process in which the multiple nodes in the multi-dimensional direct connection topology elect to determine the root node, each node in the multi-dimensional direct connection topology can send the load information of the node to each node in the multi-dimensional direct connection topology in a flooding manner, so that each node in the multi-dimensional direct connection topology can obtain the load information of each node in the multi-dimensional direct connection topology, and each node determines the root node in the multi-dimensional direct connection topology according to the received load information and the load information of the node.

[0099] The load information of any node can include the size of the storage space in the storage unit of any node or the bandwidth of any node, and the load information of any node can indicate the load capacity of any node. Optionally, when each node determines the root node in the multi-dimensional direct connection topology, the node with the strongest load capacity indicated by the load information can be determined as the root node.

[0100] In a possible implementation, before each node determines the root node in the multi-dimensional direct connection topology, each node can obtain the identifier of the node in the multi-dimensional direct connection topology. The identifier of any node can be sent to any node by a control device, or can be statically configured for any node by a human. Each node can send the identifier of the node to the nodes in the multi-dimensional direct connection topology in a flooding manner, so that each node can obtain the identifier of the node in the multi-dimensional direct connection topology.

[0101] The application does not limit the type of the identifier of any node, and the identifier of any node can be one or a combination of numbers, letters, or symbols. The identifiers of different nodes in the multi-dimensional direct connection topology are different. The identifier of any node indicates the dimension and position of any node in the multi-dimensional direct connection topology. The dimension of any node in the multi-dimensional direct connection topology refers to the level of any node in the multi-dimensional direct connection topology, and the position of any node in the multi-dimensional direct connection topology refers to the relative position of any node to a specified node in the level, and the specified node can be specified by a human.

[0102] After each node in the multi-dimensional direct connection topology obtains the identifier and the load information of each node, the node in the multi-dimensional direct connection topology can more accurately determine the root node in the multi-dimensional direct connection topology and determine the identifier of the root node. After the root node in the multi-dimensional direct connection topology is determined, the leaf node in the multi-dimensional direct connection topology transmits data to the root node, and a data reduction in the multi-dimensional direct connection topology starts.

[0103] In the data reduction process, the nodes in the multi-dimensional direct connection topology are divided into root nodes and leaf nodes, and the leaf nodes are further divided into head nodes and intermediate nodes. The head node refers to a leaf node that starts transmitting data to the root node without receiving data from other leaf nodes. The intermediate node refers to a node that transmits data of the node and data transmitted by other leaf nodes to the root node after receiving the data transmitted by other leaf nodes.

[0104] In the following, the head node is taken as the seventh node, and the nodes in the multi-dimensional direct connection topology are used for model training as an example to exemplarily describe the data reduction method performed by the head node in the data reduction process. Referring to FIG. 2, a flowchart of a data reduction method performed by the seventh node is shown, and the data reduction method performed by the seventh node includes but is not limited to the following S201 and S202.

[0105] S201, the seventh node obtains seventh data, and the seventh data is part of third generated data, and the third generated data is data generated by the seventh node in the model training process.

[0106] Exemplarily, the seventh node obtains the seventh data, including: splitting the third generated data to obtain the seventh data. The method of the seventh node for splitting the third generated data is not limited in the present application, and the seventh node can uniformly split the third generated data, or can split the third generated data according to the data characteristics of the third generated data. For example, the third generated data includes multiple types of data, and the seventh node can split the third generated data according to the multiple types in the third generated data.

[0107] The multiple data slices obtained by splitting the third generated data are part of the third generated data, and the seventh data can be any one of the multiple data slices obtained by splitting the third generated data. Optionally, the seventh data can be a data slice corresponding to a first path to which the seventh node belongs. Thus, before the seventh node obtains the seventh data, the seventh node can determine the first path to which the seventh node belongs.

[0108] In the present application, the data reduction can be data reduction according to the path, so before the nodes in the multi-dimensional direct connection topology start to perform the operation corresponding to the data reduction method, the nodes in the multi-dimensional direct connection topology can determine the path in the multi-dimensional direct connection topology, so as to perform data reduction based on the path in the multi-dimensional direct connection topology.

[0109] Based on the foregoing description, any node in the multi-dimensional direct connection topology can obtain the identifiers of all nodes in the multi-dimensional direct connection topology, and determine the root node in the multi-dimensional direct connection topology according to the identifiers of all nodes in the multi-dimensional direct connection topology. The root node is a node for performing full reduction on data generated by all nodes in the multi-dimensional direct connection topology, and the data generated by all leaf nodes in the multi-dimensional direct connection topology needs to be transmitted to the root node. Therefore, after determining the root node in the multi-dimensional direct connection topology, each leaf node can determine one or more paths from the node to the root node according to the identifiers of nodes in the multi-dimensional direct connection topology.

[0110] Taking the seventh node as an example, after determining the root node, the seventh node determines at least one path from the seventh node to the root node according to the identifiers of nodes in the multi-dimensional direct connection topology that have been obtained, and determines the path information of the at least one path according to the routing rule or the shortest path tree rule. The path information of any path includes the identifiers of all nodes on the path and the connection relationship between the nodes on the path. The relationship between any two nodes at least includes the direction of the port that implements the connection between the two nodes, for example, node 1 is connected to the northward port of node 2 through the westward port. Since the identifier of any node can indicate the position of the node, the relative positions of any two nodes can be determined through the positions of the nodes indicated by the identifiers of the nodes, and the connection relationship between the two nodes can be determined.

[0111] If the seventh node determines multiple paths, the number of nodes on the multiple paths can be the same or different. For example, referring to FIG. 3, a schematic diagram of a multi-dimensional direct connection topology provided by an embodiment of the present application is shown. The multi-dimensional direct connection topology shown in FIG. 3 is a 4*4 two-dimensional torus type topology. Each circle in FIG. 3 represents a node, and the increasing numbers on each node are the identifiers of the nodes.

[0112] The node 5 in FIG. 3 is a root node, and the other nodes are leaf nodes. After the node 15 obtains the identifiers of the nodes in the multi-dimensional direct connection topology and determines that the root node in the multi-dimensional direct connection topology is the node 5, the node 15 determines the paths from the node 15 to the node 5 according to the identifiers of the nodes in the multi-dimensional direct connection topology. For example, the four straight lines in FIG. 3 respectively represent the paths. Each node in a straight line is a node on a path. For example, the path 1 includes the node 15, the node 11, the node 7, the node 6, and the node 5, and the path 1 is connected to an east (E) interface of the node 5; the path 2 includes the node 15, the node 14, the node 13, the node 9, and the node 5, and the path 2 is connected to a south (S) interface of the node 5; the path 3 includes the node 15, the node 12, the node 8, the node 4, and the node 5, and the path 3 is connected to a west (W) interface of the node 5; and the path 4 includes the node 15, the node 3, the node 2, the node 1, and the node 5, and the path 4 is connected to a north (N) interface of the node 5.

[0113] In a multi-dimensional direct connection topology of the same specification but different types, the paths determined by the nodes are different. For example, referring to FIG. 4, another multi-dimensional direct connection topology is shown. The multi-dimensional direct connection topology shown in FIG. 4 is a 4*4 two-dimensional hyperX type topology. Each circle in FIG. 4 represents a node, and the increasing numbers on each node are the identifiers of the nodes. The node 5 in FIG. 4 is a root node, and the other nodes are leaf nodes. In the multi-dimensional direct connection topology shown in FIG. 4, the node 15 determines six paths, and the six paths are respectively shown by the straight lines in FIG. 4. Details are not described herein.

[0114] After each leaf node determines the path to which the leaf node belongs and the corresponding path information, the leaf node can send the determined path information to the root node, so that the root node determines each path in the multi-dimensional direct connection topology. For example, the seventh node can send first path information to the root node, and the first path information is used by the root node to determine a first path in the multi-dimensional direct connection topology, and the first path is any path to which the seventh node belongs.

[0115] In a possible implementation, each path in the multi-dimensional direct connection topology can be determined by the root node instead of each leaf node. After the root node determines each path, the root node sends a first packet to each leaf node in the multi-dimensional direct connection topology, and the first packet is used to indicate the information of the path in the multi-dimensional direct connection topology determined by the root node. Correspondingly, each leaf node can receive the first packet sent by the root node. Details of the content of the first packet can be found in the description of the data reduction method performed by the root node below, which is not described herein.

[0116] After receiving the first message, the seventh node can broadcast a fifth message to the nodes in the multi-dimensional direct connection topology according to the information of the path to which the seventh node belongs in the information of each path determined by the root node, the fifth message being used to indicate the information of the path in the multi-dimensional direct connection topology determined by the seventh node. Optionally, the fifth message includes at least one of the following contents:

[0117] an identifier of the first path, the first path being any path to which the seventh node belongs in the multi-dimensional direct connection topology;

[0118] an identifier of the root node on the first path;

[0119] an identifier of the seventh node;

[0120] an identifier of the eighth node, the eighth node being a previous node of the seventh node on the first path;

[0121] a message type of the fifth message.

[0122] In a possible implementation, each path in the multi-dimensional direct connection topology can constitute at least one path tree (or global tree) in the multi-dimensional direct connection topology, one path tree including a plurality of paths in the multi-dimensional direct connection topology, and the plurality of paths connecting each node in the multi-dimensional direct connection topology. For example, referring to FIG. 5, a diagram is shown for illustrating the composition of a path tree in a multi-dimensional direct connection topology according to an embodiment of the present application. The path tree in the multi-dimensional direct connection topology includes path trees in different directions, such as the northward tree, the southward tree, the eastward tree and the westward tree shown in FIG. 5, each path tree in FIG. 5 corresponding to each path in FIG. 3. In some cases, there can be a plurality of path trees in one direction. For example, referring to FIG. 6, a diagram is shown for illustrating the composition of another path tree in a multi-dimensional direct connection topology according to an embodiment of the present application, each path tree in FIG. 6 corresponding to each path in FIG. 4, wherein the number of the southward tree and the eastward tree is 2.

[0123] As can be seen from FIG. 5 and FIG. 6, in one path tree, the path to which a node belongs is unique, and thus the identifier of the path can be equivalent to the identifier of the path tree, the number of the path or the number of the path tree. Thus, the identifier of the first path in the fifth message can also be the identifier of the first path tree.

[0124] The embodiment of the present application does not limit the format of the fifth message, and the fifth message may, for example, include a tree identifier (Tree ID) field, a root node identifier (Root ID) field, a sender node identifier (Sender ID) field, a previous node identifier (PreNode ID) field, and a message type (MessageType). The content in the Tree ID field is the identifier of the path tree, i.e., the identifier of the first path to which the seventh node belongs; the content in the Root ID field is the identifier of the root node in the path tree, i.e., the identifier of the root node on the first path; the content in the Sender ID field is the identifier of the node that sends the fifth message, i.e., the identifier of the seventh node; the content in the PreNode ID field is the identifier of the previous node (which can also be referred to as the parent node of the node that sends the fifth message) of the node that sends the fifth message in the path tree (the direction is from the root node to the head node of each path), i.e., the identifier of the eighth node; and the content in the MessageType field is the message type of the fifth message, which indicates that the fifth message is a message for indicating the information of the root node and the first path.

[0125] Optionally, the fifth message may further include a next node identifier (NextNode ID) field, and the content in the NextNode ID field is the identifier of the next node of the node that sends the fifth message in the path tree (the direction is from the root node to the head node of each path), but since the seventh node is the root node on the first path, the content in the NextNode ID field in the fifth message can be none.

[0126] In a possible implementation, the nodes in the multi-dimensional direct connection topology can also be determined by the respective leaf nodes and the root node. Taking the seventh node as an example, after the seventh node determines at least one path from the seventh node to the root node, the seventh node sends first path information to the root node to indicate the at least one path determined by the seventh node. Correspondingly, other leaf nodes can also send first path information to the root node to indicate the at least one path determined by the other leaf nodes after the other leaf nodes determine the at least one path to which the other leaf nodes belong.

[0127] The root node can also determine the paths in the multi-dimensional direct connection topology according to the identifiers in the multi-dimensional direct connection topology, and determine second path information indicating the paths in the multi-dimensional direct connection topology. After receiving the first path information sent by the respective leaf nodes, the root node compares the first path information and the second path information, determines the paths in the multi-dimensional direct connection topology according to the first path information and the second path information, and generates a first message indicating the information of the paths in the multi-dimensional direct connection topology finally determined by the root node.

[0128] Exemplarily, the root node can receive the first path information sent by the first node, the first path information being used for the root node to determine the first path; obtain the second path information, the second path information indicating the path in the multi-dimensional direct connection topology determined by the root node; and in the case that the first path information and the second path information do not conflict (or are the same), broadcast the first message to the first node, the first message being used for indicating the path in the multi-dimensional direct connection topology determined by the root node.

[0129] The first path information and the second path information do not conflict, which means that the first path indicated by the first path information and the path in the multi-dimensional direct connection topology indicated by the second path information do not conflict, and thus the first path determined by the first node and the path in the multi-dimensional direct connection topology determined by the root node are both valid paths. Therefore, the root node can send the first message to the first node, so as to ensure that the first node and the root node are consistent in confirming the path in the multi-dimensional direct connection topology.

[0130] The seventh node determines the information of the paths of the seventh node by receiving the first message. In this way, the paths indicated by the first path information and the second path information are determined by the root node, so as to ensure that there is no conflict and overlap between the paths in the multi-dimensional direct connection topology finally determined by the root node.

[0131] In the embodiments of the present application, the broadcast of the first message and the fifth message can ensure that each node is consistent with the root node in recognizing the path in the multi-dimensional direct connection topology, and ensure the consistency and accuracy of the global path.

[0132] No matter which way the seventh node determines the first path to which the seventh node belongs, the seventh node can determine the seventh data corresponding to the first path. For example, the seventh node can split the third generated data according to the number of paths to which the seventh node belongs, to obtain data slices same as the number of paths to which the seventh node belongs, and determine the path corresponding to each data slice. The data slice corresponding to the first path is the seventh data. Different data slices correspond to different paths, and different paths can realize parallel transmission of data in the process of data reduction, so that the utilization rate of bandwidth in the process of data reduction in the multi-dimensional direct connection topology is high.

[0133] S202, the seventh node sends the seventh data to the root node, the seventh data being used for the root node to obtain the third data, the third data being used for the nodes in the multi-dimensional direct connection topology to perform subsequent model training.

[0134] After the seventh node obtains the seventh data, the seventh data can be sent to the root node. Optionally, if the seventh node is directly connected with the root node, the seventh node can directly send the seventh data to the root node. If the seventh node is not directly connected with the root node, the seventh node can send the seventh data to a node connected with the seventh node, and the seventh data is sent to the root node by the node connected with the seventh node. For example, the seventh node can send the seventh data to the eighth node, so that the seventh data is transmitted to the root node according to the first path.

[0135] The data reduction method performed by the head node in the multi-dimensional direct connection topology is described above by taking the seventh node as an example. Next, the data reduction method performed by the intermediate node in the multi-dimensional direct connection topology is described by taking the third node as an example. Referring to FIG. 7, a flowchart of a data reduction method performed by a third node is shown, and the data reduction method performed by the third node includes but is not limited to the following S701-S704.

[0136] S701, the third node receives fourth data, the fourth data is part of transmission data, and the transmission data is data transmitted between nodes in a model training process.

[0137] Since the third node is an intermediate node, the fourth data is data sent by an upstream node of the third node. In the embodiment of the application, in a data transmission process, the data transmitted by each node to the root node is part of the generated data of the node, and therefore the data received by the intermediate node in a data transmission process is part of the transmission data transmitted between nodes. Therefore, the fourth data received by the third node is part of the transmission data.

[0138] Based on the foregoing description, before the nodes in the multi-dimensional direct connection topology start data reduction, each node can determine a path in the multi-dimensional direct connection topology, and perform data reduction along the path according to the path to which the node belongs. In this case, before the third node receives the fourth data, the third node further sends first path information to the root node, and the first path information is used by the root node to determine the first path.

[0139] The first path information sent by the third node to the root node is obtained based on the path in the multi-dimensional direct connection topology determined by the third node. The method for the third node to determine the first path information is not limited in the embodiment of the application. For example, the method for the third node to determine the first path information can be the same as the method for the seventh node to determine the first path information.

[0140] In a possible implementation, the path in the multi-dimensional direct connection topology is determined by the root node. In this implementation, the third node can determine the path in the multi-dimensional direct connection topology by receiving the first message sent by the root node and according to the information of the path in the multi-dimensional direct connection topology determined by the root node indicated by the first message. In this way, after the third node determines the path in the multi-dimensional direct connection topology to which the third node belongs, the third node broadcasts a third message to the nodes in the multi-dimensional direct connection topology, and the third message is used to indicate the information of the path in the multi-dimensional direct connection topology determined by the third node.

[0141] The third message includes at least one of the following contents:

[0142] An identifier of a first path, the first path being any path in the multi-dimensional direct connection topology to which the third node belongs;

[0143] An identifier of a root node on the first path;

[0144] An identifier of the third node;

[0145] An identifier of a fourth node, the fourth node being a next node of the third node on the first path;

[0146] An identifier of a fifth node, the fifth node being a previous node of the third node on the first path;

[0147] A message type of the third message.

[0148] For example, the third node is node 1 in FIG. 3, the first path is a path composed of node 5, node 1, node 2, node 3 and node 15, and the identifier of the first path is Tree1. The content of the third message can be as follows.

[0149] TreeID: Tree1

[0150] RootID: 5

[0151] SenderID: 1

[0152] NextNodeID: 2

[0153] PreNodeID: 5

[0154] MessageType: RootBroadcast

[0155] In another possible implementation, after the third node autonomously determines the first path information, the third node can send the first path information to the root node, the root node can compare the first path information with the second path information autonomously generated by the root node, and then send the finally determined information of each path in the multi-dimensional direct connection topology to the third node by sending a first message to the third node. The third node can determine the information of the path to which the third node belongs according to the information of each path indicated by the first message, and generate and send a third message according to the information of the path to which the third node belongs.

[0156] No matter how the third node and other nodes in the multi-dimensional direct connection topology determine the paths in the multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology can perform data reduction according to the determined paths. In the case where the nodes in the multi-dimensional direct connection topology perform data reduction according to the paths, if the third node belongs to the first path and is an intermediate node on the first path, the fourth data received by the third node is data sent by an upstream node of the third node on the first path, and the fourth data corresponds to the first path.

[0157] S702, the third node obtains fifth data, the fifth data being part of second generated data, the second generated data being data generated by the third node in a model training process.

[0158] For example, the third node obtains the fifth data, including: splitting the second generated data to obtain the fifth data. The embodiments of the present application do not limit the method of the third node splitting the second generated data. The third node can uniformly split the second generated data, or split the second generated data according to the data characteristics of the second generated data, or split the second generated data according to the number of paths to which the third node belongs and the characteristics of the paths.

[0159] For example, the number of paths to which the third node belongs is 4. The third node can determine the load capacity of each path according to the load information of the nodes on each path, and split the second generated data according to the load capacity of each path to obtain data slices, and the fifth data is one of the data slices. One data slice corresponds to one path to which the third node belongs, and the size of the data slice corresponding to each path is positively correlated with the load capacity of each path.

[0160] In this case, the third node can store each data slice in a different storage area, and record the correspondence between the storage space and the data slice and the path, so as to accurately obtain the fifth data corresponding to the fourth data from the storage space corresponding to the path when receiving the fourth data sent by an upstream node of the third node in one path to which the third node belongs.

[0161] In a possible implementation, the fourth data is carried by a fourth packet, the fourth packet further comprising an identifier of a path corresponding to the fourth data and an identifier of the fourth data, the identifier of the path corresponding to the fourth data being used to determine the second generated data, and the identifier of the fourth data being used to determine the fifth data in the second generated data.

[0162] Referring to FIG. 8, a schematic diagram of a fourth packet is shown according to an embodiment of the present application. The fourth packet is a packet extended from a remote direct memory access over converged ethernet (RoCE) packet, and comprises an ethertype (ETH) field, an internet protocol (IP) field, a user datagram protocol (UDP) field, an infiniband base transport header (IB BTH) field, a TreeID field, a DataID field, a Message Type field, a Data field, an infiniband cyclic redundancy check (ICRC) field, and a frame check sequence (FCS) field. The TreeID field, the DataID field, and the Message Type field are fields extended in the embodiment of the present application.

[0163] Since a path to which a node belongs in a path tree is unique, the identifier of the tree in the TreeID field is the identifier of the path, and is used to identify the path corresponding to the data carried in the packet. The content in the DataID field is the identifier of the data, and is used to identify the data slice, so that the node receiving the packet can reduce or store the data to a correct position according to the content in the DataID field. The content in the Message Type field is used to identify the type of the packet. In the two processes of data reduction and sending of the reduced data by the root node to each node, the content in the Message Type field is different, and the indicated types of the packets are different, so that the node receiving the packet can perform corresponding processing and analysis on the packet according to the content in the Message Type field, so that the data to be reduced can be correctly reduced, or the reduced data can be correctly received and processed by the node.

[0164] For example, referring to FIG. 9, a schematic diagram of a fourth message provided in an embodiment of the present application is shown. The format of the fourth message in FIG. 9 is the same as that of the fourth message in FIG. 8. In the fourth message in FIG. 9, the content in the TreeID field is Tree1, which represents that the data carried by the fourth message is the data corresponding to the path in Tree1. The content in the DataID field in the fourth message is 1, which represents that the data corresponding to the slice 1 in the local generated data of the node receiving the fourth message corresponds to the data carried by the fourth message. The content in the MessageType field of the fourth message is Reduce, which represents that the type of the fourth message is the message transmitted in the process of data reduction, and indicates that the node receiving the fourth message should reduce the data carried by the fourth message and the data corresponding to the slice 1 in the local generated data.

[0165] The third node can accurately determine the second generated data corresponding to the path identifier by the path identifier and the fourth data identifier carried by the fourth message, and determine the storage area of the fifth data corresponding to the fourth data identifier in the storage area of each data slice storing the second generated data, so as to read the fifth data in the storage area of the fifth data, realize accurate acquisition of the fifth data, and ensure the accuracy and reliability of the reduction based on the fourth data and the fifth data.

[0166] Optionally, the third node can determine the storage area of the plurality of data slices corresponding to the fourth data identifier based on the fourth data identifier, determine the second generated data corresponding to the path identifier based on the path identifier corresponding to the fourth data, determine the storage area of the second generated data corresponding to the fourth data identifier in the storage area of the plurality of data slices corresponding to the fourth data identifier, and read the fifth data, to realize accurate acquisition of the fifth data. That is, in the embodiment of the present application, the order of obtaining the fifth data based on the path identifier corresponding to the fourth data and the fourth data identifier is not limited.

[0167] S703, the third node reduces the fourth data and the fifth data to obtain sixth data, and the sixth data is used to obtain the third data by the root node, and the third data is used for the node in the multi-dimensional direct connection topology to perform subsequent model training.

[0168] In the embodiment of the present application, the reduction of data can be aggregation, subtraction, multiplication, division, absolute value, etc. In a possible implementation manner, the reduction order is different when two data are reduced, and the reduction result can be different. The reduction order refers to the order of operation on data in the process of data reduction.

[0169] For example, if the reduction of the two data is to add the two data, the node determines the type of the result according to the type of the data of the first operation in the process of adding the two data. If the type of the data of the first operation is a floating point number, the type of the result is determined as a floating point number even if the type of the data of the second operation is an integer. If the type of the data of the first operation is an integer, the type of the result is determined as an integer regardless of the type of the data of the second operation.

[0170] For example, if the data of the first operation is 0.1 and the type is a floating point number, and the data of the second operation is 100 and the type is an integer, the result of the addition is 100.1 and the type of the result is a floating point number. If the data of the first operation is 100 and the type is an integer, and the data of the second operation is 0.1, the result of the addition is 100 and the type of the result is an integer.

[0171] Optionally, since the reduction order has an impact on the result of the reduction of the two data, in the embodiment of the present application, when the two data are reduced, the reduction can be performed in a specified order to ensure that the reduction result meets the reduction requirement. For example, the fourth data and the fifth data are reduced to obtain the sixth data, including: reducing the fourth data and the fifth data according to a second reduction order to obtain the sixth data, and the second reduction order is determined based on the data transmission order on the path corresponding to the fourth data and the fifth data. The second reduction order can be the same as the data transmission order on the path corresponding to the fourth data and the fifth data, or it can be opposite to the data transmission order.

[0172] Taking the reduction of the fourth data and the fifth data as an example, if the second reduction order is the same as the data transmission order, since the fourth data is transmitted before the fifth data on the path corresponding to the fourth data and the fifth data, the sixth data is the sum of the fourth data and the fifth data. If the second reduction order is opposite to the data transmission order, the sixth data is the sum of the fifth data and the fourth data.

[0173] In the embodiment of the present application, each time the data is reduced, the reduction can be performed in a specified order to ensure that the results obtained by reducing the same data at different times are the same, to realize the reproducibility of the reduction result, to meet the demand of the associativity of the large model training or reasoning, and to ensure the accuracy of the reduction result.

[0174] In a possible implementation, the third node can determine to reduce the fourth data and the fifth data based on the computing resources on the third node. For example, the third node can process the fourth data and the fifth data according to the following two cases according to whether the computing resources of the third node meet the resource condition.

[0175] Case 1: In a case where the computing resource of the third node meets the resource condition, the fourth data and the fifth data are reduced to obtain the sixth data, and the resource condition is used to indicate that the computing resource of the third node is greater than the resource required for reducing the fourth data and the fifth data.

[0176] The computing resource of the third node can refer to various resources for calculation on the third node, including the calculation space of the third node or the computing power of the third node, etc. The computing power of the third node refers to the ability of the third node to perform numerical calculation and processing tasks.

[0177] If the computing resource on the third node meets the resource condition when the third node obtains the fourth data and the fifth data, it means that the computing power of the third node is sufficient to reduce the fourth data and the fifth data, so the third node can reduce the fourth data and the fifth data to obtain the sixth data, and send the sixth data to the downstream node of the third node on the path corresponding to the sixth data.

[0178] If the computing resource on the third node does not meet the resource condition when the third node obtains the fourth data and the fifth data, it means that the computing power of the third node is insufficient to reduce the fourth data and the fifth data, so the third node can temporarily not reduce the fourth data and the fifth data, but wait until part of the computing resource on the third node is released and the computing resource of the third node meets the resource condition, then reduce the fourth data and the fifth data to obtain the sixth data, and send the sixth data to the downstream node of the third node on the path corresponding to the sixth data.

[0179] Case 2: In a case where the computing resource of the third node does not meet the resource condition, the fourth data and the fifth data are sent to the sixth node, the resource condition is used to indicate that the computing resource of the third node is greater than the resource required for reducing the fourth data and the fifth data, and the sixth node is the previous node of the third node on the path corresponding to the fourth data.

[0180] If the computing resource on the third node does not meet the resource condition when the third node obtains the fourth data and the fifth data, the third node can not reduce the fourth data and the fifth data, and does not wait for part of the computing resource on the third node to be released, but directly sends the fourth data and the fifth data to the downstream node of the third node on the path corresponding to the fourth data and the fifth data, and the downstream node reduces the fourth data, the fifth data and part of the data of the downstream node.

[0181] In a possible implementation, in a case where the third node does not reduce the fourth data and the fifth data and directly sends the fourth data and the fifth data to a downstream node, the third node can further send a seventh message to the downstream device before sending the fourth data and the fifth data to the downstream node, to indicate that the third node does not reduce the fourth data and the fifth data. The downstream node, upon receiving the seventh message sent by the third node, determines that the data sent by the third node needs to include the fourth data and the fifth data, and performs reduction on the fourth data, the fifth data, and part of the generated data of the downstream node after confirming that the fourth data and the fifth data have been received, to ensure the accuracy of the reduction.

[0182] Since the upstream node of the third node can also be in a case where the computing resource does not meet the resource condition, the fourth data received by the third node can be data reduced by the upstream node or data not reduced by the upstream node. If the fourth data received by the third node is data reduced by the upstream node, the third node can reduce the fourth data and the fifth data in a case where the computing resource of the third node meets the resource condition after receiving the message carrying the fourth data. If the fourth data is data not reduced by the upstream node and the fourth data includes a plurality of sub-data, the third node can reduce the plurality of sub-data included in the fourth data and the fifth data after receiving a plurality of messages carrying the plurality of sub-data in the fourth data.

[0183] In a possible implementation, the third node can open storage spaces for the data to be received and the generated data of the third node. The present application does not limit the size of the storage space opened by the third node for each data and the number of storage spaces opened by the third node. For example, the third node can determine the number of storage spaces to be opened according to the number of paths to which the third node belongs in the full reduction process. For example, if the third node belongs to 4 paths in the full reduction process, the third node can open 2 storage spaces for each path, that is, the third node can open 8 storage spaces in total. One of the two storage spaces corresponding to one path is used to store the data sent by the upstream node on the path, and the other storage space is used to store part of the second generated data of the third node corresponding to the path.

[0184] Based on the foregoing description, it can be known that the fourth data received by the third node from the upstream node can be unreduced data, and the fourth data can include data carried by multiple messages. In this case, after the third node receives the seventh message sent by the upstream node, the third node can allocate more storage spaces for the fourth data based on the seventh message, so as to store multiple sub-data in the fourth data in the multiple storage spaces respectively. Alternatively, the seventh message can include the number of messages carrying each sub-data of the fourth data, so that the third node can accurately allocate multiple storage spaces for the fourth data based on the number in the seventh message, to avoid resource waste.

[0185] In the embodiments of the present application, the combination of the sequential reduction and the mechanism of pre-allocating storage spaces for the to-be-received data can further ensure the accuracy of data reduction. For example, referring to FIG. 10, a schematic diagram of data reduction provided by an embodiment of the present application is shown. IO Die in the figure is a device for data reduction on the third node. The IO Die pre-allocates storage spaces for the to-be-received remote data (i.e. data sent by other nodes to the third node, such as the fourth data) in the cache, and allocates cache spaces for the to-be-reduced local data (such as the fifth data). When the IO Die on the third node receives the remote data, the remote data b+c is stored in the corresponding storage space, the to-be-reduced local data a is stored in the storage space, and the local data and the remote data are reduced according to the second reduction order, to obtain the reduced data a+b+c. The second reduction order is the order of local data+remote data, that is, the local data is taken as the first operation data, and the remote data is taken as the second operation data.

[0186] FIG. 10 shows the process of data reduction of the third node in the case that the remote data is the reduced data of the upstream node, and the third node pre-allocates storage spaces for the reduced data of the upstream node. In some cases, the fourth data received by the third node can be unreduced data, and the fourth data includes multiple sub-data. In this case, the third node can determine the number of sub-data included in the fourth data according to the received seventh message, and increase the storage spaces allocated for the fourth data, to ensure that the third node can sequentially reduce each sub-data included in the fourth data and the fifth data after receiving each sub-data, to ensure the accuracy of the reduction result.

[0187] For example, referring to FIG. 11, a schematic diagram of data reduction is shown according to an embodiment of the present application. The difference between FIG. 11 and FIG. 10 is that the third node receives the seventh message, and the seventh message indicates that the fourth data includes two sub-data, remote data 1 (data b) and remote data 2 (data c), so the third node increases the number of storage spaces opened for the fourth data, so that the IO Die of the third node performs reduction on the sub-data of the fourth data and the fifth data according to the second reduction order after receiving all the sub-data of the fourth data.

[0188] In this case, if the second reduction order is the order of local data + remote data 1 + remote data 2, even if the third node receives remote data 2 first, the third node will not reduce the local data and remote data 2, but will wait to receive remote data 1, and then reduce the local data, remote data 1 and remote data 2 according to the reduction order to obtain the reduction result a+b+c, ensuring the accuracy of the reduction result.

[0189] S704, the third node sends the sixth data to the root node.

[0190] Since the third node is an intermediate node, after the third node reduces the fourth data and the fifth data to obtain the sixth data, the third node still needs to send the sixth data to the root node, so that the root node can obtain the global reduction data according to the sixth data.

[0191] If the third node is directly connected to the root node, the third node can directly send the sixth data to the root node. If the third node is not directly connected to the root node, the third node can send the sixth data to the node connected to the third node, and the node connected to the third node sends the sixth data to the root node. For example, the third node can send the sixth data to the fourth node, so that the third node is transmitted to the root node according to the first path.

[0192] The above describes the data reduction method performed by the intermediate node in the multi-dimensional direct connection topology from S701 to S704, and the data reduction method performed by the root node in the multi-dimensional direct connection topology is exemplarily described below. Referring to FIG. 12, a flowchart of a data reduction method performed by a root node is shown according to an embodiment of the present application, and the data reduction method performed by the root node includes but is not limited to the following S1201 to S1204.

[0193] S1201, the root node receives first data, the first data being part of transmission data, the transmission data being data transmitted between nodes in a model training process.

[0194] The first data is data sent by an upstream node of the root node. In the embodiments of the present application, in the process of one data transmission, each node transmits part of the generated data of the node to the root node, and thus the root node receives part of the transmission data transmitted between the nodes in the process of one data transmission. Therefore, the first data received by the root node is part of the transmission data.

[0195] Based on the foregoing description, before the nodes in the multi-dimensional direct connection topology start data reduction, each node can determine the path in the multi-dimensional direct connection topology and perform data reduction along the path according to the path to which the node belongs. The embodiments of the present application do not limit the manner in which the root node determines the nodes in the multi-dimensional direct connection topology before the root node receives the first data.

[0196] For example, the root node receives first path information sent by a first node, and the first path information is used by the root node to determine the first path. The first node is a leaf node in the multi-dimensional direct connection topology, including but not limited to the third node and the seventh node in the foregoing description. After determining the first path information, the first node sends the first path information to the root node. Since the first path information indicates the information of the path in the multi-dimensional direct connection topology determined by the first node, the root node can determine the path in the multi-dimensional direct connection topology determined by the first node according to the first path information, and take the path in the multi-dimensional direct connection topology determined by the first node as the path in the multi-dimensional direct connection topology determined by the root node.

[0197] In a possible implementation, the root node obtains second path information, the second path information indicating the path in the multi-dimensional direct connection topology determined by the root node; and broadcasts a first message to the first node, the first message being used to indicate the information of the path in the multi-dimensional direct connection topology determined by the root node.

[0198] Since the root node can obtain the identifier of the node in the multi-dimensional direct connection topology, and the identifier in the multi-dimensional direct connection topology indicates the dimension and position of the node in the multi-dimensional direct connection topology, the root node can determine the path in the multi-dimensional direct connection topology based on the identifier of the node in the multi-dimensional direct connection topology, thereby obtaining the second path information.

[0199] Alternatively, the path in the multi-dimensional direct connection topology can be manually specified, and the root node is configured with the second path information, so as to obtain the second path information. After determining the second path information, the root node broadcasts the first message to the first node, so that the first node can determine the information of the path determined by the root node based on the first message.

[0200] The first message includes at least one of the following contents:

[0201] An identifier of the first path, the first path being any path in the multi-dimensional direct connection topology;

[0202] an identifier of the root node;

[0203] an identifier of the node sending the first message;

[0204] an identifier of the second node, the second node being a node following the root node on the first path;

[0205] a message type of the first message.

[0206] In the case that the root node is node 5 in FIG. 3, the first path is a path composed of node 5, node 1, node 2, node 3 and node 15, and the identifier of the first path is Tree1, the content of the first message can be as follows.

[0207] TreeID: Tree1

[0208] RootID: 5

[0209] SenderID: 5

[0210] NextNodeID: 1

[0211] PreNodeID: None

[0212] MessageType: RootBroadcast

[0213] In another possible implementation, the root node can acquire the first path information and the second path information, and compare the first path information and the second path information to determine the paths in the multi-dimensional direct connection topology.

[0214] In this way, the paths indicated by the first path information and the second path information are determined by the root node, so that the paths in the multi-dimensional direct connection topology determined by the root node do not conflict or overlap. The paths in the multi-dimensional direct connection topology determined by the root node.

[0215] Regardless of the way in which the root node and other nodes in the multi-dimensional direct connection topology determine the paths in the multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology can perform data reduction according to the determined paths. In the case that the nodes in the multi-dimensional direct connection topology perform data reduction according to the paths, if the root node belongs to the first path, the first data received by the root node is data sent by a node upstream of the root node on the first path (i.e., the first node on the first path), and the first data corresponds to the first path.

[0216] S1202, the root node obtains second data, the second data being part of the first generated data, the first generated data being data generated by the root node in a model training process.

[0217] Exemplarily, the root node obtaining the second data comprises: splitting the first generated data to obtain the second data. Embodiments of the present application do not limit the method of the root node splitting the first generated data. The root node can uniformly split the first generated data, or split the first generated data according to data characteristics of the first generated data, or split the first generated data according to the number of paths to which the root node belongs and path characteristics.

[0218] Taking the number of paths to which the root node belongs as 4 as an example, the root node can determine the load capacity of each path according to the load information of nodes on each path, and split the first generated data according to the load capacity of each path to obtain data slices, the second data being one of the data slices. One data slice corresponds to one path to which the root node belongs, and the size of the data slice corresponding to each path is positively correlated with the load capacity of each path.

[0219] In this case, the root node can store each data slice in different storage areas, and record the correspondence between the storage space and the data slice and the path, so as to accurately obtain the second data corresponding to the first data from the storage space corresponding to the path when receiving the first data sent by the upstream node of the root node in one path to which the root node belongs.

[0220] In a possible implementation, the first data is carried by a second packet, the second packet further comprising an identifier of the path corresponding to the first data and an identifier of the first data, the identifier of the path corresponding to the first data being used to determine the first generated data, and the identifier of the first data being used to determine the second data in the first generated data.

[0221] The format of the second packet can be the same as that of the fourth packet, and the root node determines the second data according to the identifier of the path corresponding to the first data and the identifier of the first data. The process can refer to the process of the third node determining the fifth data according to the identifier of the path corresponding to the fourth data and the identifier of the fourth data in S702, which will not be repeated here.

[0222] S1203, the root node reduces the first data and the second data to obtain third data, the third data being used for nodes in the multi-dimensional direct connection topology to perform subsequent model training.

[0223] In a possible implementation, the reducing the first data and the second data to obtain the third data comprises: reducing the first data and the second data according to a first reducing order to obtain the third data, and the first reducing order is determined based on a data transmission order on a path corresponding to the first data and the second data. The first reducing order can be the same as the data transmission order on the path corresponding to the first data and the second data, or can be opposite to the data transmission order.

[0224] In S1204, the root node sends the third data to a first node in the multi-dimensional direct connection topology, and the first node is a node in the multi-dimensional direct connection topology except the root node.

[0225] The embodiments of the present application do not limit the manner in which the root node sends the third data to the first node in the multi-dimensional direct connection topology. For example, if the root node is directly connected to the first node, the root node can directly send the third data to the first node. If the root node is not directly connected to the first node and the first node is a node on the first path, the root node can return the third data to the first node according to the first path. The root node sends the third data to the first node, so that the first node can perform subsequent model training based on the reduction result, and ensure that the model training is successfully completed.

[0226] In a possible implementation, since the nodes in the multi-dimensional direct connection topology perform data reduction according to the multiple paths in the multiple path trees, the root node can receive data sent by an upstream node of the root node on each path to which the root node belongs, and reduce the received data and part of the first generated data according to the path corresponding to the data, to obtain multiple reduction results. The third data is a reduction result corresponding to one path.

[0227] Therefore, the root node can not only send the third data to the first node, but also send reduction results corresponding to other paths to the first node. The embodiments of the present application do not limit the manner in which the root node sends the reduction results corresponding to other paths to the first node. For example, the root node can send the reduction results corresponding to each path to the first node on each path according to each path; or the root node can also pack the multiple reduction results and send the packed results to the first node.

[0228] Optionally, the root node can further reduce the multiple reduction results corresponding to the multiple paths to obtain a full reduction result, and directly send the full reduction result to the first node or send the full reduction result to the first node according to any path to which the first node belongs.

[0229] Referring to FIG. 13, a complete flow diagram of data reduction is shown according to an embodiment of the present application. Node 1 and node 2 are examples of leaf nodes in a multi-dimensional direct connection topology. In step 1, each node (including the root node, node 1 and node 2) is assigned a node identifier, and each node exchanges the node identifier with each other node through flooding. In step 2, the root node broadcasts information of the path in the multi-dimensional direct connection topology determined by the root node to node 1 and node 2, to ensure that each node has consistent knowledge of the path in the multi-dimensional direct connection topology and the root node. In step 3, the leaf nodes start reducing towards the root node along the path, and the root node broadcasts the final reduction result back to the leaf nodes.

[0230] In the process of data reduction, two tasks of communication and calculation need to be executed in parallel. In the execution of the communication task, data needs to be read from the storage unit of the node and transmitted, and the bandwidth of the node used for transmitting data is occupied. In the execution of the calculation task, data needs to be read from the storage unit, and since the calculation unit and the storage unit in the node are different units, data also needs to be transmitted inside the node in the process of executing the calculation task, and the bandwidth of the node used for transmitting data is also occupied. Therefore, it can be seen that in data reduction, the two tasks of communication and calculation compete for the storage unit and the bandwidth of the node, and this competition leads to a decline in the communication performance of the node.

[0231] In the embodiments of the present application, the module with forwarding function connected to the node or deployed on the node can be brushed with new software or program through soft modification, so that the module with forwarding function can not only receive and send data, but also reduce data, realizing the function of edge transmission and edge calculation. For example, referring to FIG. 14, a schematic diagram of a node is shown according to an embodiment of the present application, the node includes DDR, calculation Die and IO Die, and the IO Die is the module with forwarding function on the node, and the in-network computing engine for data reduction is brushed on the IO Die.

[0232] Through soft modification of the IO Die, when the IO Die receives data of other nodes (as shown in ① of FIG. 14), the IO Die reads the data to be reduced on the node stored in the DDR (as shown in ② of FIG. 14), and reduces and transmits the data to be reduced on the node and the received data of other nodes (as shown in ③ of FIG. 14), realizing the offloading calculation of the IO Die, reducing the number of calls to the DDR, and reducing the competition between the two tasks of calculation and communication for the storage unit and the bandwidth in the process of data reduction, thereby improving the efficiency of communication.

[0233] In addition, in the process of one data reduction in the embodiment of the application, the static time delay between the start of data transmission from the head node to the determination of the reduction result by the root node is small. For example, referring to FIG. 15, a schematic diagram of a time delay provided by the embodiment of the application is shown. In the two-dimensional 4*4 Torus type topology shown in FIG. 3, the maximum hop number is 5 in the multi-tree all-reduce, and the time delay that cannot be covered by the pipeline is 5 packet (or packet, pkt) transmission time delays. For example, when node 12 transmits pkt1 to pkt5, node 5 cannot receive pkt and cannot work, but when node 12 transmits pkt6, node 5 can receive pkt1 and start working. In the direct connection topology, the bandwidth is full, the transmission times are reduced by half, the static time delay and the DDR in-out times are reduced, and the network communication efficiency and the bandwidth utilization are effectively improved, and the linear growth of the static time delay and the memory access overhead of the hierarchical ring all-reduce in the direct connection topology with the increase of the nodes is avoided.

[0234] In the multi-dimensional direct connection topology shown in FIG. 3, the total time delay in the data reduction process is (α+θ)+2(M / D·1 / B+L_(non-over)), where D is the degree of the node in the multi-dimensional direct connection topology, that is, the number of nodes directly connected to any node. D in the multi-dimensional direct connection topology shown in FIG. 3 is 4. α is the static overhead, θ is the memory overhead, B is the single-link bandwidth. L_(non-over)=(MaxHop×PktSize) / B considers that the hops of different nodes to the root node are different (for example, the maximum hop number in the multi-dimensional direct connection topology shown in FIG. 3 is 5, and the maximum hop number in the multi-dimensional direct connection topology shown in FIG. 4 is 3), and it is assumed that the nodes in the multi-dimensional direct connection topology can perform data reduction along the pipeline, and the value should be the static time delay that cannot be covered.

[0235] Next, the related art and the embodiment of the application are compared, and the beneficial effects of the embodiment of the application are further described and supplemented.

[0236] Referring to FIG. 16, an implementation scenario diagram of a related art is provided. The related art is applied to a two-dimensional hyperX type topology of 4*4, including 16 nodes, and any one of the 16 nodes can be connected to each node in the same dimension. The related art achieves high throughput by pipeline embedding, and a root node is arranged on each row (for example, y0 to y3 direction in the figure) in the related art, and the root nodes on each row are located in the same column, and a root node of the entire topology is further arranged on the column to which the root nodes on each row belong. As shown in FIG. 16, the root nodes on each row in x0 to x3 are located in the y2 column, and the node with coordinates (x2, y2) in the y2 column is determined as the root node of the entire topology.

[0237] In the process of data reduction, each node transmits local data to the root node on the row to which the node belongs, the root node on each row sends the received data and local data to the root node of the entire topology, and the root node of the entire topology performs reduction on the data of each node in the entire topology. However, in the related art, each node sends complete local data to the root node in the process of data reduction, and a part of the available network bandwidth in the topology is used, and the utilization rate of the bandwidth is low.

[0238] For example, referring to FIG. 17, an operation flowchart is shown when the related art performs data reduction. The node in the related art includes three parts of a computing Die, a DDR, and an IO Die, which are connected to each other. The computing Die includes a communication core and a computing core, the communication core is used to transmit data, and the computing core is used to perform reduction and other calculations on the data. When the node shown in FIG. 17 performs data reduction, the IO Die on the node receives data sent by other nodes, and transmits the received data of other nodes to the computing Die (as shown by ① in FIG. 17). The computing Die reads local data stored in the DDR through the communication core (as shown by ② in FIG. 17), and performs reduction on the local data and the received data of other nodes through the computing core to obtain reduced data. Then, the computing Die writes the reduced data back to the DDR (as shown by ③ in FIG. 17), so that the DDR stores the reduced data. When the node needs to send the reduced data to other nodes, the computing Die reads the reduced data stored in the DDR through the communication core, and transmits the reduced data to the IO Die, and the IO Die obtains the reduced data and sends the reduced data to other nodes (as shown by ④ in FIG. 17).

[0239] In the process of data reduction in the related art, the DDR needs to be called three times, the data in the DDR needs to be read twice, and the data needs to be written to the DDR once, and in the process of calling the DDR, data transmission in the node is also needed, which causes the communication task and the calculation task to compete for the storage unit DDR and the communication bandwidth of the node, thereby reducing the communication efficiency.

[0240] In summary, the data reduction method provided by the embodiments of the present application can improve the communication efficiency and the utilization rate of the bandwidth in the process of data reduction.

[0241] The data reduction method provided in the embodiments of the present application is introduced above. Corresponding to the above method, the embodiments of the present application further provide a data reduction device. The device is applied to a root node. The device is used to execute the data reduction method performed by the root node in FIG. 12 through various modules shown in FIG. 18. As shown in FIG. 18, the data reduction device provided in the embodiments of the present application includes the following modules.

[0242] A data reduction device is provided. The device is applied to at least one root node in a multi-dimensional direct connection topology. The nodes in the multi-dimensional direct connection topology are used to perform model training. The device includes: a first transceiver module 1801, configured to receive first data. The first data is part of transmission data. The transmission data is data transmitted between nodes in a model training process. A first processing module 1802 is configured to obtain second data. The second data is part of first generated data. The first generated data is data generated by the root node in the model training process. The first data and the second data are reduced to obtain third data. The third data is used for the nodes in the multi-dimensional direct connection topology to perform subsequent model training. The first transceiver module 1801 is configured to send the third data to a first node in the multi-dimensional direct connection topology. The first node is a node in the multi-dimensional direct connection topology except the root node.

[0243] In a possible implementation, the first data is data corresponding to a first path in the multi-dimensional direct connection topology. The first node is a node on the first path. The first transceiver module 1801 is further configured to receive first path information sent by the first node. The first path information is used for the root node to determine the first path.

[0244] In a possible implementation, the first processing module 1802 is further configured to obtain second path information. The second path information indicates a path in the multi-dimensional direct connection topology determined by the root node. The first transceiver module 1801 is further configured to broadcast a first packet to the first node. The first packet is used to indicate information of the path in the multi-dimensional direct connection topology determined by the root node.

[0245] In a possible implementation, the first packet includes at least one of the following contents: an identifier of the first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of the root node; an identifier of a node sending the first packet; an identifier of a second node, the second node being a node after the root node on the first path; a packet type of the first packet.

[0246] In a possible implementation, the first data is carried by a second packet. The second packet further includes an identifier of a path corresponding to the first data and an identifier of the first data. The identifier of the path corresponding to the first data is used to determine the first generated data. The identifier of the first data is used to determine the second data in the first generated data.

[0247] In a possible implementation, the first processing module 1802 is configured to split the first generated data to obtain second data.

[0248] In a possible implementation, the first processing module 1802 is configured to perform reduction on the first data and the second data according to a first reduction order to obtain third data, where the first reduction order is determined based on a data transmission order on a path corresponding to the first data and the second data.

[0249] Referring to FIG. 19, a data reduction apparatus is provided, which is applied to a third node. The third node is a node in a multi-dimensional direct connection topology except for a root node. The nodes in the multi-dimensional direct connection topology are configured to perform model training. The apparatus includes a second transceiver module 1901 configured to receive fourth data. The fourth data is part of transmission data. The transmission data is data transmitted between nodes in a model training process. The apparatus includes a second processing module 1902 configured to obtain fifth data. The fifth data is part of second generated data. The second generated data is data generated by the third node in the model training process. The apparatus includes a reduction performed on the fourth data and the fifth data to obtain sixth data. The sixth data is used by the root node to obtain third data. The third data is used by the nodes in the multi-dimensional direct connection topology to perform subsequent model training. The second transceiver module 1901 is configured to send the sixth data to the root node.

[0250] In a possible implementation, the fourth data is data corresponding to a first path in the multi-dimensional direct connection topology. The third node is a node on the first path. The second transceiver module 1901 is further configured to send first path information to the root node. The first path information is used by the root node to determine the first path.

[0251] In a possible implementation, the second transceiver module 1901 is further configured to receive a first message sent by the root node. The first message is used to indicate information of a path in the multi-dimensional direct connection topology determined by the root node. The second transceiver module 1901 is further configured to broadcast a third message to the nodes in the multi-dimensional direct connection topology. The third message is used to indicate information of a path in the multi-dimensional direct connection topology determined by the third node.

[0252] In a possible implementation, the third message includes at least one of the following contents: an identifier of the first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of a root node on the first path; an identifier of the third node; an identifier of a fourth node, the fourth node being a node after the third node on the first path; an identifier of a fifth node, the fifth node being a node before the third node on the first path; and a message type of the third message.

[0253] In a possible implementation, the fourth data is carried by a fourth packet, the fourth packet further includes an identifier of a path corresponding to the fourth data and an identifier of the fourth data, the identifier of the path corresponding to the fourth data is used to determine the second generated data, and the identifier of the fourth data is used to determine the fifth data in the second generated data.

[0254] In a possible implementation, the second processing module 1902 is configured to split the second generated data to obtain the fifth data.

[0255] In a possible implementation, the second processing module 1902 is configured to perform reduction on the fourth data and the fifth data according to a second reduction order to obtain the sixth data, and the second reduction order is determined based on a data transmission order on a path corresponding to the fourth data and the fifth data.

[0256] In a possible implementation, the second processing module 1902 is configured to perform reduction on the fourth data and the fifth data to obtain the sixth data, in a case where the computing resource of the third node meets a resource condition, and the resource condition is used to indicate that the computing resource of the third node is greater than a resource required for reduction on the fourth data and the fifth data.

[0257] In a possible implementation, the second transceiver 1901 is configured to send the fourth data and the fifth data to a sixth node, in a case where the computing resource of the third node does not meet a resource condition, and the resource condition is used to indicate that the computing resource of the third node is greater than a resource required for reduction on the fourth data and the fifth data, and the sixth node is a previous node of the third node on a path corresponding to the fourth data.

[0258] Referring to FIG. 20, a data reduction apparatus is provided, and the apparatus is applied to a seventh node, the seventh node is a node in a multi-dimensional direct connection topology except a root node, and the nodes in the multi-dimensional direct connection topology are used to perform model training, the apparatus includes: a third processing module 2002 configured to obtain seventh data, the seventh data is part of third generated data, and the third generated data is data generated by the seventh node in a model training process; and a third transceiver 2001 configured to send the seventh data to the root node, and the seventh data is used for the root node to obtain third data, and the third data is used for the nodes in the multi-dimensional direct connection topology to perform subsequent model training.

[0259] In a possible implementation, the seventh node is a node on a first path in the multi-dimensional direct connection topology; and the third transceiver 2001 is further configured to send first path information to the root node, and the first path information is used for the root node to determine the first path.

[0260] In a possible implementation, the third receiving module 2001 is further configured to receive a first message sent by the root node, the first message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the root node; and broadcast a fifth message to nodes in the multi-dimensional direct connection topology, the fifth message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the seventh node.

[0261] In a possible implementation, the fifth message includes at least one of the following: an identifier of a first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of a root node on the first path; an identifier of the seventh node; an identifier of an eighth node, the eighth node being a previous node of the seventh node on the first path; and a message type of the fifth message.

[0262] In a possible implementation, the third processing module 2002 is configured to split the third generated data to obtain seventh data.

[0263] In a possible implementation, the first processing module, the second processing module, or the third processing module 2002 is further configured to obtain identifiers of nodes in the multi-dimensional direct connection topology, the identifier of any node indicating a dimension and a position of the any node in the multi-dimensional direct connection topology; and determine a path in the multi-dimensional direct connection topology and at least one root node according to the identifiers of the nodes in the multi-dimensional direct connection topology.

[0264] In a possible implementation, the at least one root node is determined based on load information of the nodes in the multi-dimensional direct connection topology.

[0265] It should be understood that the apparatus provided by the above implementation has the same beneficial effects as the data reduction method, which will not be described herein again. In addition, the apparatus, when implementing its functions, is only taken as an example for the division of the above functional modules, and in actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus and the method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be described herein again.

[0266] In an example embodiment, a data reduction system is provided, which includes a root node, a third node, and a seventh node, the root node is configured to execute the data reduction method in FIG. 12, the third node is configured to execute the data reduction method in FIG. 7, and the seventh node is configured to execute the data reduction method in FIG. 2.

[0267] In an example embodiment, there is provided a data reduction device, the device comprising a processor coupled with a memory; the memory having stored therein at least one instruction, the at least one instruction being loadable and executable by the processor to cause the data reduction device to implement the data reduction method of Figure 2, or to implement the data reduction method of Figure 7, or to implement the data reduction method of Figure 12.

[0268] In an example embodiment, there is provided a computer program (product), the computer program (product) comprising: computer program code which, when run on a computer, causes the computer to perform the method of Figure 2, Figure 7 or Figure 12.

[0269] In an example embodiment, there is provided a computer readable storage medium storing a program or instructions, the program or instructions being run on a computer to cause the method of Figure 2, Figure 7 or Figure 12 to be performed.

[0270] In an example embodiment, there is provided a chip comprising a processor for calling and running instructions stored in a memory to cause a computer on which the chip is installed to perform the method of Figure 2, Figure 7 or Figure 12.

[0271] In an example embodiment, there is provided another chip comprising: an input interface, an output interface, a processor and a memory, the input interface, the output interface, the processor and the memory being connected through internal connection paths, the processor being configured to execute code in the memory, the code being executed to cause a computer on which the chip is installed to perform the method of Figure 2, Figure 7 or Figure 12.

[0272] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer, all or part generates the processes or functions described in the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another via wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as Solid State Disk) and the like.

[0273] The terms "first", "second", and the like in the present application are used to distinguish between elements or items having substantially the same function and similar items, and it should be understood that there is no logical or chronological dependency between "first", "second", "nth", and the quantity and execution order are not limited. It should also be understood that although the following description uses the terms first, second, and the like to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another.

[0274] It should also be understood that in various embodiments of the present application, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0275] In the present application, the term "at least one" means one or more, and the term "multiple" in the present application means two or more, for example, multiple second devices means two or more second devices. The terms "system" and "network" are often used interchangeably in this document.

[0276] It should be understood that the words “example” and “exemplary” are used herein to mean serving as an instance, example, or illustration, and not as a limitation. The terms “comprising,” “including,” and “having” are intended to be open-ended terms that mean one or more of the stated components are present, but not excluding additional components. The terms “a,” “an,” and “the” are intended to be interpreted to include both singular and plural forms, unless otherwise indicated by context.

[0277] It should also be understood that the term “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term “and / or” is a descriptive term that refers to an associated relationship between items in a list. It is intended that the term “and / or” be interpreted to mean that three possible relationships can exist among the items in the list. For example, A and / or B can mean that A exists alone, B exists alone, or both A and B exist together.

[0278] It should also be understood that the terms “if’ and “when” can be interpreted to mean “upon” or “in response to a determination” or “in response to a detection.” Similarly, the phrase “if determined” or “if detected [a stated condition or event]” can be interpreted to mean “upon a determination” or “in response to a determination” or “upon a detection” or “in response to a detection” of [a stated condition or event], depending on the context.

[0279] The above description is merely exemplary in nature of the application, and is not intended to limit the present disclosure, application, or uses.

Claims

1. A method of data reduction, comprising: The method is applied to at least one root node in a multi-dimensional direct connection topology, nodes in the multi-dimensional direct connection topology are used for model training, and the method comprises: receiving first data, the first data being part of transmission data, the transmission data being data transmitted between nodes in the model training process; obtaining second data, the second data being part of first generated data, the first generated data being data generated by the root node in the model training process; reducing the first data and the second data to obtain third data, the third data being used for subsequent model training of nodes in the multi-dimensional direct connection topology; sending the third data to a first node in the multi-dimensional direct connection topology, the first node being a node other than the root node in the multi-dimensional direct connection topology.

2. The method of claim 1, wherein, The first data is data corresponding to a first path in the multi-dimensional direct connection topology, and the first node is a node on the first path; before receiving the first data, the method further comprises: receiving first path information sent by the first node, the first path information being used for the root node to determine the first path.

3. The method according to claim 1 or 2, characterized in that, Before receiving the first data, the method further comprises: obtaining second path information, the second path information indicating a path in the multi-dimensional direct connection topology determined by the root node; broadcasting a first packet to the first node, the first packet being used to indicate information of the path in the multi-dimensional direct connection topology determined by the root node.

4. The method of claim 3, wherein, The first packet comprises at least one of the following contents: an identifier of a first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of the root node; an identifier of a node sending the first packet; an identifier of a second node, the second node being a node subsequent to the root node on the first path; a packet type of the first packet.

5. The method according to any of claims 1 to 4, characterized in that, The first data is carried by a second packet, the second packet further comprising an identifier of a path corresponding to the first data and an identifier of the first data, the identifier of the path corresponding to the first data being used to determine the first generated data, and the identifier of the first data being used to determine the second data in the first generated data.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: segmenting the first generated data to obtain the second data.

7. The method according to any of claims 1 to 6, characterized in that The method further comprises: reducing the first data and the second data according to a first reduction order to obtain the third data, the first reduction order being determined based on a data transmission order on a path corresponding to the first data and the second data.

8. A data reduction method characterized by, The method is applied to a third node, the third node being a node other than a root node in a multi-dimensional direct connection topology, nodes in the multi-dimensional direct connection topology being used for model training, and the method comprising: receiving fourth data, the fourth data being part of transmission data, the transmission data being data transmitted between nodes in the model training process; obtaining fifth data, the fifth data being part of second generated data, the second generated data being data generated by the third node in the model training process; reducing the fourth data and the fifth data to obtain sixth data, the sixth data being used by the root node to obtain third data, the third data being used by nodes in the multi-dimensional direct connection topology to perform subsequent model training; sending the sixth data to the root node.

9. The method of claim 8, wherein, The fourth data is data corresponding to a first path in the multi-dimensional direct connection topology, and the third node is a node on the first path. Before the fourth data is received, the method further includes: sending first path information to the root node, the first path information being used by the root node to determine the first path.

10. The method according to claim 8 or 9, characterized in that, Before the fourth data is received, the method further includes: receiving a first message sent by the root node, the first message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the root node; broadcasting a third message to nodes in the multi-dimensional direct connection topology, the third message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the third node.

11. The method of claim 10, wherein, The third message includes at least one of the following contents: an identifier of a first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of a root node on the first path; an identifier of the third node; an identifier of a fourth node, the fourth node being a next node of the third node on the first path; an identifier of a fifth node, the fifth node being a previous node of the third node on the first path; a message type of the third message.

12. The method according to any of claims 8-11, characterized by, The fourth data is carried by a fourth message, the fourth message further including an identifier of a path corresponding to the fourth data and an identifier of the fourth data, the identifier of the path corresponding to the fourth data being used to determine the second generated data, and the identifier of the fourth data being used to determine the fifth data in the second generated data.

13. The method according to claims 8-12, characterized in that, The obtaining of the fifth data includes: segmenting the second generated data to obtain the fifth data.

14. The method of any one of claims 8-13, wherein, The reducing of the fourth data and the fifth data to obtain the sixth data includes: reducing the fourth data and the fifth data according to a second reduction order to obtain the sixth data, the second reduction order being determined based on a data transmission order on a path corresponding to the fourth data and the fifth data.

15. The method of any of claims 8-14, wherein, The reducing of the fourth data and the fifth data to obtain the sixth data includes: in a case where a computing resource of the third node meets a resource condition, reducing the fourth data and the fifth data to obtain the sixth data, the resource condition being used to indicate that the computing resource of the third node is greater than a resource required for reducing the fourth data and the fifth data.

16. The method of any one of claims 8-14, wherein, After the fourth data is received, the method further includes: In a case where the computing resource of the third node does not satisfy a resource condition, the fourth data and the fifth data are sent to a sixth node, the resource condition is used to indicate that the computing resource of the third node is greater than the resource required for reducing the fourth data and the fifth data, and the sixth node is a previous node of the third node on a path corresponding to the fourth data.

17. A method of data reduction, comprising: The method is applied to a seventh node, the seventh node is a node in a multi-dimensional direct connection topology except a root node, and the nodes in the multi-dimensional direct connection topology are used for model training. The method comprises: obtaining seventh data, the seventh data being part of third generated data, the third generated data being data generated by the seventh node in the model training process; sending the seventh data to the root node, the seventh data being used for the root node to obtain third data, and the third data being used for the nodes in the multi-dimensional direct connection topology to perform subsequent model training.

18. The method of claim 17, wherein, The seventh node is a node on a first path in the multi-dimensional direct connection topology. Before the seventh data is obtained, the method further comprises: sending first path information to the root node, the first path information being used for the root node to determine the first path.

19. The method of claim 17 or 18, wherein, Before the seventh data is obtained, the method further comprises: receiving a first message sent by the root node, the first message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the root node; broadcasting a fifth message to the nodes in the multi-dimensional direct connection topology, the fifth message being used to indicate information of a path in the multi-dimensional direct connection topology determined by the seventh node.

20. The method of claim 19, wherein, The fifth message comprises at least one of the following contents: an identifier of a first path, the first path being any path in the multi-dimensional direct connection topology; an identifier of a root node on the first path; an identifier of the seventh node; an identifier of an eighth node, the eighth node being a previous node of the seventh node on the first path; a message type of the fifth message.

21. The method of any of claims 17-20, wherein, The seventh data is obtained by: segmenting the third generated data to obtain the seventh data.

22. The method of any one of claims 1-21, wherein, The method further comprises: obtaining identifiers of the nodes in the multi-dimensional direct connection topology, an identifier of any node indicating a dimension and a position of the any node in the multi-dimensional direct connection topology; determining the paths in the multi-dimensional direct connection topology and the at least one root node according to the identifiers of the nodes in the multi-dimensional direct connection topology.

23. The method of any one of claims 1-22, wherein, The at least one root node is determined based on load information of the nodes in the multi-dimensional direct connection topology.

24. A data reduction device, comprising: The device is applied to at least one root node in a multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology are used for model training, and the device comprises: a first transceiving module configured to perform the receiving and / or sending related operations in the method of any one of claims 1-7, 22 or 23; a first processing module configured to perform other operations in addition to the receiving and / or sending related operations in the method of any one of claims 1-7, 22 or 23.

25. A data reduction device, comprising: The device is applied to a third node, the third node being a node other than a root node in a multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology being used for model training, and the device comprising: The second transceiving module is configured to perform the receiving and / or sending related operations in the method of any one of claims 8-16, 22 or 23. The second processing module is configured to perform operations other than the receiving and / or sending related operations in the method of any one of claims 8-16, 22 or 23.

26. A data reduction device, comprising: The device is applied to a seventh node, the seventh node being a node other than a root node in a multi-dimensional direct connection topology, the nodes in the multi-dimensional direct connection topology being used for model training, and the device comprising: The third transceiving module is configured to perform the receiving and / or sending related operations in the method of any one of claims 17-23. The third processing module is configured to perform operations other than the receiving and / or sending related operations in the method of any one of claims 17-23.

27. A data reduction device, comprising: The device comprises a processor coupled with a memory, and the memory stores at least one instruction loaded and executed by the processor, so that the data reduction device implements the data reduction method of any one of claims 1-23.

28. A data reduction system, comprising: The system comprises a root node, a third node and a seventh node, the root node being configured to perform the data reduction method of any one of claims 1-7, 22 or 23, the third node being configured to perform the data reduction method of any one of claims 8-16, 22 or 23, and the seventh node being configured to perform the data reduction method of any one of claims 17-23.

29. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction loaded and executed by the processor to implement the data reduction method of any one of claims 1-23.

30. A computer program product, characterised in that, The computer program product comprises computer programs / instructions executed by the processor to enable the computer to implement the data reduction method of any one of claims 1-23.

Images