A data aggregation method and device based on ring topology network

Abstract

The present application discloses a data aggregation method based on a ring topology network, the method comprising: at any node side in the ring topology network, the node remotely aggregates the first data obtained from the node to the second data in the next neighbor node of the node; or the node remotely aggregates the aggregation result of the local aggregation of the third data from the previous neighbor node of the node and the first data obtained from the node to the second data in the next neighbor node of the node. The present application reduces the time required for the ring aggregation operation and improves the efficiency of the ring aggregation operation.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data aggregation method based on a ring topology network. Background Art

[0002] The ring topology network consists of network nodes connected in a fixed direction to form a closed loop. Each node is connected to its adjacent previous neighbor node and next neighbor node. Each device can be directly connected to the ring as a node, or connected to the ring through an interface device and branch cables.

[0003] In a ring topology network for artificial intelligence, nodes include a central processing unit (CPU) and an accelerator card, where the accelerator card includes but is not limited to a graphics processing unit (GPU), a TPU (Tensor Processing Unit), an NPU (Neural Network Processing Unit), a DPU (Deep Learning Processing Unit), an APU (Accelerated Processing Unit), an FPGA, and a GPGPU (General-Purpose computing on Graphics Processing Unit). The ring topology network can be composed of a single node with multiple accelerator cards, or multiple nodes with multiple accelerator cards.

[0004] Data aggregation is used to store the aggregated results obtained by calculating the data in multiple processes according to the set mapping function in one process. Data aggregation based on the ring topology network can be used to aggregate the data in the first process distributed in each node of the ring topology structure in the second process of each node, that is, each node in the ring topology network has the same aggregated result. As the number of nodes in the ring topology network increases, the efficiency of data aggregation based on the ring topology network will decrease. Summary of the invention

[0005] The present invention provides a data aggregation method based on a ring topology network to improve the data aggregation efficiency of an accelerator card.

[0006] A first aspect of the present invention provides a data aggregation method based on a ring topology network, the method comprising:

[0007] On the local node side with any node in the ring topology network as the local node,

[0008] The local node remotely aggregates the first data obtained from the local node to the second data in the next neighboring node of the local node;

[0009] or,

[0010] The local node performs local aggregation on the third data from the previous neighbor node of the local node and the first data obtained from the local node, and remotely aggregates the aggregation result to the second data in the next neighbor node of the local node.

[0011] Preferably, the number of the nodes is more than 2, and the local node remotely aggregates the first data obtained from the local node to the second data in the next neighbor node of the local node, including:

[0012] In the first working sequence, the local node obtains the first data from the local node,

[0013] In the second working sequence, the local node remotely aggregates the acquired first data to the second data in the next neighboring node of the local node.

[0014] Preferably, the local node performs local aggregation on the third data from the previous neighbor node of the local node and the first data obtained from the local node, and remotely aggregates the aggregation result to the second data in the next neighbor node of the local node, including:

[0015] In the first working sequence, the node obtains the third data from the previous neighbor node of the node.

[0016] In the second working sequence, the local node obtains the first data from the local node,

[0017] In the third working sequence, the node performs a local aggregation operation on the first data and the third data to obtain an aggregation result.

[0018] In the fourth working sequence, the node remotely aggregates the aggregation result to the second data in the next neighboring node of the node.

[0019] Preferably, the method further comprises:

[0020] The node aggregates the third data from the previous neighbor node of the node with the fourth data in the node so that the fourth data is remotely aggregated by the third data.

[0021] The fifth data in the next neighbor node is acquired by the next neighbor node and remotely aggregated to the sixth data in the next neighbor node of the next neighbor node,

[0022] The seventh data of the previous neighbor node is remotely aggregated with the eighth data from the previous neighbor node of the previous neighbor node.

[0023] Preferably, the first working sequence further includes:

[0024] In the first working sequence, the previous neighbor node of the local node obtains the third data in the previous neighbor node, and the next neighbor node obtains the fifth data in the next neighbor node;

[0025] Preferably, the second working sequence further includes:

[0026] In the second working sequence, this node aggregates the third data from the previous neighbor node of this node with the fourth data in this node, and the next neighbor node remotely aggregates the acquired fifth data to the sixth data in the next neighbor node of the next neighbor node, and the seventh data of the previous neighbor node is remotely aggregated with the eighth data from the previous neighbor node of the previous neighbor node.

[0027] Preferably, the first working sequence further includes:

[0028] In the first working sequence, all nodes in the ring topology network respectively obtain first data from their local locations;

[0029] Preferably, the second working sequence further includes:

[0030] In the second working sequence, all nodes in the ring topology network aggregate the first data they obtain into the second data in their next neighbor node.

[0031] Preferably, the number of nodes is 4 or more; the previous neighbor node aggregates the aggregation result from the previous neighbor node of the previous neighbor node with the seventh data in the previous neighbor node so that the seventh data is remotely aggregated by the aggregation result, and the fifth data in the next neighbor node is acquired by the next neighbor node of the next neighbor node;

[0032] in,

[0033] When the number of nodes is 4, the previous neighbor node of the previous neighbor node and the next neighbor node of the next neighbor node are the same node.

[0034] Preferably, the first working sequence further includes:

[0035] In the first working sequence, the fifth data in the next neighbor node is acquired by the next neighbor node of the next neighbor node, and the previous neighbor node of the previous neighbor node acquires the ninth data of its previous neighbor node;

[0036] Preferably, the second working sequence further includes:

[0037] In the second working sequence, the next neighbor node of the next neighbor node obtains the sixth data locally, and the previous neighbor node of the previous neighbor node obtains the eighth data locally;

[0038] Preferably, the third working sequence further includes:

[0039] In the third working sequence, the previous neighbor node of the previous neighbor node performs a local aggregation operation on the eighth data and the ninth data, and the next neighbor node of the next neighbor node performs a local aggregation operation on the fifth data and the sixth data;

[0040] Preferably, the fourth working sequence further includes:

[0041] In the fourth working sequence, the next neighbor node of the next neighbor node remotely aggregates the aggregation result to its next neighbor node, and the previous neighbor node of the previous neighbor node remotely aggregates the aggregation result to the previous neighbor node.

[0042] Preferably, the first working sequence further includes:

[0043] In the first working sequence, all nodes in the ring topology network respectively obtain third data from their upper neighbor nodes;

[0044] Preferably, the second working sequence further includes:

[0045] In the second working sequence, all nodes in the ring topology network respectively obtain the first data from their local locations;

[0046] Preferably, the third working sequence further includes:

[0047] In the third working sequence, all nodes in the ring topology network respectively perform local aggregation operations on the first data and the third data.

[0048] Preferably, the fourth working sequence further includes:

[0049] In the fourth working sequence, all nodes in the ring topology network respectively remotely aggregate the aggregation result to the second data in the next neighbor node of the local node.

[0050] Preferably, the node is a single-node with multiple accelerator cards or a multi-node with multiple accelerator cards.

[0051] Preferably, the remote aggregation is performed in the following manner:

[0052] The arithmetic control unit in the L2 cache of the accelerator card reads the local data in the L2 cache, reads the remote data from the neighbor in the L2 cache, performs operations on the read local data and remote data, and writes the aggregated results to the cache location where the local data is located;

[0053] The local aggregation is performed as follows:

[0054] The computing core of the accelerator card reads local data in the cache, reads remote data from neighboring nodes in the cache, operates on the read local data and remote data, obtains aggregated results and transmits them, and the cache includes at least one of an input cache, a secondary cache, and a high-bandwidth cache.

[0055] A second aspect of the present application provides a data aggregation device based on a ring topology network, wherein the nodes are connected to form a ring topology network.

[0056] The nodes include:

[0057] a storage circuit for storing data to be aggregated, and

[0058] a processor, connected to the storage circuit,

[0059] Wherein, the processor is used to:

[0060] Remotely aggregate the first data obtained from the local node to the second data in the next neighboring node of the local node;

[0061] or,

[0062] The aggregation result of locally aggregating the first data obtained from the local node and the third data from the previous neighboring node of the local node is remotely aggregated to the second data in the next neighboring node of the local node.

[0063] The data aggregation method based on the ring topology network provided by the embodiment of the present application reduces the steps of local aggregation and the step of writing the aggregation result to the next neighbor node by remotely aggregating the first data obtained from the local node to the second data in the next neighbor node of the local node, thereby reducing the time required for the ring aggregation operation and improving the efficiency of the ring aggregation operation; by remotely aggregating the aggregation result of local aggregation of the third data from the previous neighbor node of the local node and the first data obtained from the local node to the second data in the next neighbor node of the local node, the number of people involved in the ring aggregation operation is reduced, and the time required for the ring aggregation operation is reduced, which not only improves the communication efficiency of the ring topology network, but also improves the performance of the communication operator. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of a data aggregation method based on a ring topology network in the related art.

[0065] Figure 2 A schematic diagram of a ring aggregation operation performed on all accelerator cards in a ring topology network in the related art.

[0066] Figure 3a A schematic diagram of a data aggregation method based on a ring topology network according to an embodiment of the present application.

[0067] Figure 3b Another schematic diagram of a data aggregation method based on a ring topology network according to an embodiment of the present application.

[0068] Figure 4 A schematic diagram of a data aggregation method based on a ring topology network according to an embodiment of the present application.

[0069] Figure 5 A schematic diagram of remote aggregation performed by the accelerator card in this embodiment.

[0070] Figure 6 A schematic diagram of a full aggregation operation of summing in the first embodiment of the present invention.

[0071] Figure 7 This is a schematic diagram of a data aggregation method based on a ring topology network according to the second embodiment of the present application.

[0072] Figure 8 A schematic diagram of a full aggregation operation for summing in the second embodiment. DETAILED DESCRIPTION

[0073] In order to make the objectives, technical means and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings.

[0074] See also Figure 1 As shown, Figure 1 A schematic diagram of a data aggregation method based on a ring topology network in the related art. In the figure, the ring topology network includes 4 nodes, each of which includes an accelerator card, which includes a computing core (CU core) for calculation, an input buffer for caching first data, and a staging buffer for caching second data. The gray block in the accelerator card in the figure represents the staging buffer, and the light blue block represents the input buffer.

[0075] The applicant has found that in the data aggregation method based on the ring topology network, for each accelerator card in the ring topology network, the accelerator card needs four steps to complete a data aggregation. Taking any accelerator card as an example, the accelerator card performs the following steps:

[0076] At the first working sequence t1, the first step of reading the first data from the input buffer of the accelerator card is shown by the red line in the figure.

[0077] In the second working sequence t2, the second step of reading the second data from the temporary cache of the accelerator card is shown by the green line in the figure.

[0078] At the third working sequence t3, the first data and the second data are aggregated (reduce) to obtain the third step of the aggregation result, as shown by the pink line in the figure.

[0079] In the fourth working sequence t4, the fourth step is to write the aggregation result into the temporary cache of the next neighboring accelerator card of the current accelerator card, and, in this working sequence, the aggregation result from the previous neighboring accelerator card of the current accelerator card is written into the temporary cache of the current accelerator card, as shown by the blue line in the figure.

[0080] In the figure, the same color lines used to represent the execution steps also indicate that the accelerator cards work in parallel at the same working sequence. It can be seen that for an accelerator card, the duration of a reduce operation is the sum of t1, t2, t3, and t4.

[0081] Take a ring topology network with 4 GPUs as an example to illustrate the ring all reduce operation. Figure 2 As shown, Figure 2 A schematic diagram of a ring aggregation operation performed on all accelerator cards in a ring topology network in the related art.

[0082] At timing 1, GPU0 writes a0 to GPU1, GPU1 writes b1 to GPU2, GPU2 writes c2 to GPU3, and GPU3 writes d3 to GPU0;

[0083] At time sequence 2, GPU1 writes its aggregation result a0+a1 to GPU2, GPU2 writes its aggregation result b1+b2 to GPU3, GPU3 writes its aggregation result c3+c2 to GPU0, and GPU0 writes its aggregation result d3+d0 to GPU1;

[0084] At time sequence 3, GPU2 writes its aggregation result a0+a1+a2 to GPU3, GPU3 writes its aggregation result b1+b2+b3 to GPU0, GPU0 writes its aggregation result c3+c2+c0 to GPU1, and GPU1 writes its aggregation result d3+d0+d1 to GPU2.

[0085] At timing 4, GPU3 writes its aggregation result a3+a2+a1+a0 to GPU0, GPU0 writes its aggregation result b0+b3+b2+b1 to GPU1, GPU1 writes its aggregation result c1+c0+c3+c2 to GPU2, and GPU2 writes d2+d1+d0+d3 to GPU3.

[0086] At time sequence 5, GPU0 writes its aggregation result a3+a2+a1+a0 to GPU1, GPU1 writes its aggregation result b0+b3+b2+b1 to GPU2, GPU2 writes its aggregation result c1+c0+c3+c2 to GPU3, and GPU3 writes its aggregation result d2+d1+d0+d3 to GPU0.

[0087] At time sequence 6, GPU1 writes its aggregation result a3+a2+a1+a0 to GPU2, GPU2 writes its aggregation result b0+b3+b2+b1 to GPU3, GPU3 writes its aggregation result c1+c0+c3+c2 to GPU0, and GPU0 writes its aggregation result d2+d1+d0+d3 to GPU1.

[0088] At time sequence 7, GPU2 obtains the aggregation result a3+a2+a1+a0, GPU3 obtains the aggregation result c1+c0+c3+c2, GPU0 obtains the aggregation result d2+d1+d0+d3, and GPU1 obtains the aggregation result d2+d1+d0+d3.

[0089] At this point, all GPUs save the same aggregation results, thus completing a ring all reduce operation.

[0090] It can be seen that it takes 7 timings for all GPUs in the ring topology network to complete the ring all reduce operation. Among them, timings 1 to 4 are used for the full aggregation (all reduce) operation of one data in all accelerator cards in the ring topology network, that is, timings 1 to 4 are used for the all reduce operation of one ring data, and timings 5 ​​to 7 are used for broadcast operations. Assuming that the broadcast operation has the same working timing as the reduce operation, the duration required for the ring all reduce operation is: 7×(t1+t2+t3+t4).

[0091] Without loss of generality, if the number of nodes in the ring topology network is n, the time required for the ring all reduce operation is: (2n-1)×(t1+t2+t3+t4). As n increases, the time required for the ring all reduce operation also increases. Not only is the data aggregation efficiency of each node not high enough, but all nodes in the ring topology network participate in data aggregation.

[0092] In view of this, an embodiment of the present application provides a data aggregation method based on a ring topology network to improve data aggregation efficiency.

[0093] See also Figure 3a As shown, Figure 3aThis is a schematic diagram of a data aggregation method based on a ring topology network according to an embodiment of the present application. The ring topology network includes more than two nodes, each of which includes an input cache and a computing core. The blue square in the node in the figure represents the input cache, and the CU square represents the computing core. The method includes:

[0094] On any node side in a ring topology network,

[0095] The node remotely aggregates the first data obtained from the node to the second data in the next neighbor node of the node.

[0096] As an example,

[0097] In the first working sequence, the local node obtains the first data from the local node, such as Figure 3a For example, the computing core in this node loads or reads data from the input cache of this node.

[0098] In the second working sequence, the node remotely aggregates the acquired first data to the second data in the next neighbor node of the node, such as Figure 3a The green arrows in the figure represent remote aggregations.

[0099] As an example, the method further includes:

[0100] The node aggregates the third data from the previous neighbor node of the node with the fourth data in the node, so that the fourth data is remotely aggregated by the third data.

[0101] The fifth data in the next neighbor node of the current node is acquired by the next neighbor node and remotely aggregated to the sixth data in the next neighbor node of the next neighbor node.

[0102] The seventh data of the previous neighbor node of the current node is remotely aggregated with the eighth data from the previous neighbor node of the previous neighbor node.

[0103] In an embodiment of the present application, when the current node obtains data from itself and directly performs aggregation operations with the data in the next neighboring node of the current node, since the aggregation operation is not performed in the current node itself but in the next neighboring node, the aggregation operation is equivalent to being performed remotely relative to the current node, so it is called remote aggregation.

[0104] See also Figure 3b As shown, Figure 3bAnother schematic diagram of the data aggregation method based on the ring topology network of the embodiment of the present application. The ring topology network includes more than 3 nodes, each node includes an input cache and a computing core. The blue square in the node in the figure represents the input cache, and the CU square represents the computing core. The local node aggregates the first data obtained from the local node with the third data from the previous neighbor node of the local node, and remotely aggregates the second data in the next neighbor node of the local node.

[0105] As an example,

[0106] In the first working sequence, the node obtains the third data from the previous neighbor node of the node, such as Figure 3b The blue arrow in the

[0107] In the second working sequence, the local node obtains the first data from the local node, such as Figure 3b For example, the red arrow in the node indicates that the computing core in this node loads or reads data from the input buffer of this node.

[0108] In the third working sequence, the node aggregates the first data and the third data to obtain an aggregation result, such as Figure 3b The pink line in

[0109] In the fourth working sequence, the node remotely aggregates the aggregation result to the second and third data in the next neighbor node of the node, such as Figure 3b The green arrows in the figure represent remote aggregations.

[0110] As an example, the method further includes:

[0111] The previous neighbor node of this node locally aggregates the aggregation result from the previous neighbor node of the previous neighbor node and the seventh data in the previous neighbor node, and the fifth data in the next neighbor node is acquired by the next neighbor node of the next neighbor node.

[0112] In an embodiment of the present application, when the local node directly aggregates its own aggregation result with the data in the next neighbor node of the local node, given that the aggregation operation is not performed in the local node itself but in the next neighbor node, the aggregation operation is equivalent to being performed remotely relative to the local node, so it is called remote aggregation; when the local node aggregates the data from itself and the data from the neighboring nodes in the local node to obtain its own aggregation result, given that the aggregation operation is performed by the computing core in the local node itself, it is called local aggregation.

[0113] It should be understood that the aggregation operation in the present application may be the integration of data, including but not limited to reduction operations such as summation and maximum value.

[0114] To facilitate understanding of the embodiments of the present application, a ring topology network composed of four acceleration cards is used as an example for explanation below.

[0115] Embodiment 1

[0116] See also Figure 4 As shown, Figure 4 This is a schematic diagram of a data aggregation method based on a ring topology network according to an embodiment of the present application. For any accelerator card in the ring topology network, perform the following steps:

[0117] At the first working sequence t1, the accelerator card loads the first data from the input buffer of the accelerator card, that is, the accelerator card reads the local data.

[0118] At the second working sequence t2, the accelerator card aggregates the first data to the input cache of the next neighbor accelerator card of the accelerator card to aggregate it with the second data in the input cache of the next neighbor accelerator card, that is, the accelerator card performs remote aggregation, and at this working sequence, the accelerator card aggregates the third data read from the input cache of the previous neighbor accelerator card of the accelerator card to the fourth data in the input cache of the accelerator card, so that the fourth data is remotely aggregated with the third data.

[0119] See also Figure 5 As shown, Figure 5 A schematic diagram of remote aggregation performed by the acceleration card of this embodiment. The acceleration card includes i computing cores and a secondary cache, where i is a natural number not less than 1. The arithmetic logic unit (ALU) in the secondary cache reads remote data, such as the third data, transmitted to the secondary cache (L2 cache) via the high-bandwidth memory (HBM), and reads existing data, i.e., local data, such as the first data, in the secondary cache. The arithmetic logic unit aggregates the first data and the third data and then writes them to the secondary cache, such as writing them to the cache location of the existing data, to reduce the occupied space of the cache.

[0120] See also Figure 6 As shown, Figure 6 A schematic diagram of a ring all reduce operation for summing in the first embodiment.

[0121] At time sequence 1, GPU0 aggregates a0 to GPU1, and the aggregation result of GPU1 is a0+a1. GPU1 aggregates b1 to GPU2, and the aggregation result of GPU2 is b1+b2. GPU2 aggregates c2 to GPU3, and the aggregation result of GPU3 is c2+c3. GPU3 aggregates d3 to GPU0, and the aggregation result of GPU0 is d3+d0.

[0122] At time sequence 2, GPU1 aggregates a0+a1 to GPU2, and the aggregation result of GPU2 is a0+a1+a2. GPU2 aggregates b1+b2 to GPU3, and the aggregation result of GPU3 is b1+b2+b3. GPU3 aggregates c2+c3 to GPU0, and the aggregation result of GPU0 is c2+c3+c0. GPU0 aggregates d3+d0 to GPU1, and the aggregation result of GPU1 is d3+d0+d1.

[0123] At time sequence 3, GPU2 aggregates a0+a1+a2 to GPU3. At this time, the aggregation result of GPU3 is a0+a1+a2+a3. GPU3 aggregates b1+b2+b3 to GPU0. At this time, the aggregation result of GPU0 is b1+b2+b3+b0. GPU0 aggregates c2+c3+c0 to GPU1. At this time, the aggregation result of GPU1 is c2+c3+c0+c1. GPU1 aggregates d3+d0+d1 to GPU2. At this time, GPU2 is d3+d0+d1+d2.

[0124] At sequence 4, GPU3 aggregates a0+a1+a2+a3 to GPU0. Since the aggregation operation of a0+a1+a2+a3 has been completed in advance, GPU3 will write it to GPU0. Similarly, GPU0 writes b1+b2+b3+b0 to GPU1, GPU1 writes c2+c3+c0+c1 to GPU2, and GPU2 writes d3+d0+d1+d2 to GPU3.

[0125] At timing 5, similarly, since the aggregation operation of a0+a1+a2+a3 has been completed in advance, GPU0 writes a0+a1+a2+a3 to GPU1, GPU1 writes b1+b2+b3+b0 to GPU2, GPU2 writes c2+c3+c0+c1 to GPU3, and GPU3 writes d3+d0+d1+d2 to GPU0.

[0126] At sequence 6, similarly, since the aggregation operation of a0+a1+a2+a3 has been completed, GPU1 writes a0+a1+a2+a3 to GPU2, GPU2 writes b1+b2+b3+b0 to GPU3, GPU3 writes c2+c3+c0+c1 to GPU0, and GPU0 writes d3+d0+d1+d2 to GPU1.

[0127] At time sequence 7, GPU2 obtains the aggregation result a3+a2+a1+a0, GPU3 obtains the aggregation result c1+c0+c3+c2, GPU0 obtains the aggregation result d2+d1+d0+d3, and GPU1 obtains the aggregation result d2+d1+d0+d3.

[0128] At this point, all GPUs save the same aggregation results, thus completing a ring all reduce operation.

[0129] In this embodiment, all accelerator cards in the ring topology network need 7 time sequences to complete a ring all reduce operation, of which 4 time sequences are used for the all reduce operation of one data in all accelerator cards in the ring topology network, and 3 time sequences are used for the broadcast operation. Then, the time required for a ring all reduce operation is: 7×(t1+t2).

[0130] Without loss of generality, if the number of accelerator cards in the ring topology network is n, the time required for the ring all reduce operation is: (2n-1)×(t1+t2). Compared with the existing technology, the time required to complete the ring all reduce operation is reduced, and the communication bandwidth and communication operator performance of the accelerator card are improved.

[0131] Embodiment 2

[0132] See also Figure 7 As shown, Figure 7 This is a schematic diagram of a data aggregation method based on a ring topology network in Embodiment 2 of the present application. For any accelerator card in the ring topology network, perform the following steps:

[0133] At the first working sequence t1, the accelerator card obtains the third data from the previous neighbor accelerator card of the accelerator card, which is equivalent to remotely reading the third data, and at the working sequence, the seventh data of the input cache of the previous neighbor accelerator card is remotely aggregated with the aggregation result of the previous neighbor accelerator card of the previous neighbor accelerator card.

[0134] At the second working sequence t2, the accelerator card reads the first data from the input buffer of the accelerator card.

[0135] At the third working sequence t3, the accelerator card performs a local aggregation operation on the first data and the third data.

[0136] In the fourth working sequence t4, the accelerator card aggregates the result of the aggregation operation to the input cache of the next neighbor accelerator card of the accelerator card to perform an aggregation operation with the third data in the input cache of the next neighbor accelerator card, which is equivalent to a remote aggregation operation, and in this working sequence, the fifth data in the input cache of the next neighbor accelerator card is read by the next neighbor accelerator card of the next neighbor accelerator card, which is equivalent to remote reading.

[0137] The process of remote aggregation operation in the above steps is similar to Figure 5 The same is true to improve the efficiency of the polymerization operation.

[0138] The local aggregation operation in the above steps can be performed by the computing core. As an example, the computing core reads local data in the cache, such as the first data, reads remote data from a neighboring node in the cache, such as the third data, performs operations on the read local data and remote data, obtains aggregation results and transmits them for subsequent remote aggregation. The cache includes at least one of an input cache, a secondary cache, and a high-bandwidth cache.

[0139] In this embodiment, the previous neighbor accelerator card of the previous neighbor accelerator card of the current accelerator card and the previous next neighbor accelerator card of the next neighbor accelerator card of the current accelerator card are the same accelerator card.

[0140] See also Figure 8 As shown, Figure 8 A schematic diagram of a ring all reduce operation for summing in the second embodiment.

[0141] At time sequence 1, GPU2 aggregates the aggregation result b1+b2 of b1 in GPU1 and b2 in GPU2 to GPU3. At this time, the aggregation result of GPU3 is b3+b2+b1. GPU1 aggregates the aggregation result c1+ c0 of c1 in GPU1 and c0 in GPU0 to GPU2. At this time, the aggregation result of GPU2 is c1+ c0+c2. GPU0 aggregates the aggregation result d0+d3 of d0 in GPU0 and d3 in GPU3 to GPU1. At this time, the aggregation result of GPU1 is d0+d3+d1. GPU3 aggregates the aggregation result a2+a3 of a3 in GPU3 and a2 in GPU2 to GPU0. At this time, the aggregation result of GPU0 is a2+a3+a0.

[0142] At sequence 2, GPU1 aggregates the aggregation result a2+a3+a0+a1 of a2+a3+a0 in GPU0 and a1 in GPU1 to GPU2. Since GPU2 has completed the aggregation operation in advance, GPU2 will not perform the aggregation operation and write the aggregation result a2+a3+a0+a1 to GPU2; GPU3 aggregates the aggregation result b3+b2+b1+b0 of b3+b2+b1 in GPU3 and b0 in GPU0 to GPU1. Since GPU1 has completed the aggregation operation in advance, GPU1 will not perform the aggregation operation and write the aggregation result b3+b2+b1+b0 to GPU 1; GPU3 aggregates the aggregation result c1+c0+c2+c3 of c1+c0+c2 in GPU2 and c3 in GPU3 to GPU0. Since GPU0 has completed the aggregation operation in advance, GPU0 will not perform the aggregation operation and write the aggregation result c1+c0+c2+c3 to GPU0; GPU2 aggregates the aggregation result d0+d3+d1+d2 of d0+d3+d1 in GPU1 and d2 in GPU2 to GPU3. Since GPU3 has completed the aggregation operation in advance, GPU3 will not perform the aggregation operation and write the aggregation result d0+d3+d1+d2 to GPU3.

[0143] At timing 3, since the aggregation operation of d0+d3+d1+d2 in GPU0 has been completed in advance, GPU0 writes d0+d3+d1+d2 in GPU3 into GPU0; similarly, GPU3 writes a2+a3+a0+a1 in GPU2 into GPU3, GPU2 writes b3+b2+b1+b0 in GPU1 into GPU2, and GPU1 writes c1+c0+c2+c3 in GPU0 into GPU1.

[0144] At this point, all GPUs save the same aggregation results, thus completing a ring all reduce operation.

[0145] In this embodiment, it takes three time sequences for all accelerator cards in the ring topology network to complete a ring all reduce operation. Assuming that the write time sequence is regarded as an aggregation operation, the time required for a ring all reduce operation is: 3×(t1+t2+t3+t4).

[0146] Without loss of generality, if the number of accelerator cards in the ring topology network is n, the time required for a ring all reduce operation is: (n-1)×(t1+t2+t3+t4).

[0147] Compared to Figure 2In the prior art solution, four time sequences are required to implement the all reduce operation of a ring data, and only one accelerator card has the result of the all reduce operation of the ring data. In this embodiment, the aggregation of a ring data is simplified to two time sequences, and two accelerator cards already have the aggregation result of the ring data, which reduces the timing and steps of the broadcast operation, and is conducive to improving the communication efficiency and performance of the ring topology network; in addition, only two accelerator cards are required to participate in the ring all reduce operation of this embodiment, for example, Figure 7 In the example, GPU3 can complete the aggregation of d0~d3, and GPU1 can complete the aggregation of b0~b3. GPU0 and GPU2 can take on other tasks while GPU3 and GPU1 are performing the ring all reduce operation. Without loss of generality, for n accelerator cards, only n / 2 accelerator cards need to participate, where n is an even number greater than 3.

[0148] The embodiment of the present application also provides a data aggregation device based on a ring topology network, wherein the nodes are connected to form a ring topology network.

[0149] The nodes include:

[0150] a storage circuit for storing data to be aggregated, and

[0151] a processor, connected to the storage circuit,

[0152] Wherein, the processor is used to:

[0153] Remotely aggregate the first data obtained from the local node to the second data in the next neighboring node of the local node;

[0154] or,

[0155] The aggregation result of locally aggregating the first data obtained from the local node and the second data from the previous neighbor node of the local node is remotely aggregated to the third data in the next neighbor node of the local node.

[0156] The storage circuit may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0157] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0158] As for the apparatus / network-side device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0159] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0160] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data aggregation method based on a ring topology network, characterized in that: A plurality of accelerator cards are connected to form a ring topology network, and the method includes: On any accelerator card in a ring topology network, The accelerator card remotely aggregates the first data acquired by the accelerator card to the second data in the next neighbor accelerator card of the accelerator card, so as to perform one aggregation; or, The accelerator card locally aggregates the third data from the previous neighbor accelerator card of the accelerator card and the first data obtained from the accelerator card, and remotely aggregates the data to the second data in the next neighbor accelerator card of the accelerator card, so as to perform one aggregation; in, The remote aggregation is used to accelerate the arithmetic control unit in the secondary cache of the card to perform operations on the data in the secondary cache. The local aggregation is used to accelerate the computing core in the card to operate on the data in the cache, wherein the cache includes at least one of an input cache, a secondary cache, and a high-bandwidth cache; The local accelerator card remotely aggregates first data acquired from the local accelerator card to second data in a next neighbor accelerator card of the local accelerator card, including: The accelerator card obtains first data from the accelerator card, The accelerator card remotely aggregates the acquired first data into second data in a next neighbor accelerator card of the accelerator card; The local accelerator card performs local aggregation on third data from a previous neighbor accelerator card of the local accelerator card and first data obtained from the local accelerator card, and remotely aggregates the aggregation result to second data in a next neighbor accelerator card of the local accelerator card, including: The accelerator card obtains the third data from the accelerator card that is the previous neighbor of the accelerator card. The accelerator card obtains first data from the accelerator card, The accelerator card performs a local aggregation operation on the first data and the third data to obtain an aggregation result, The accelerator card remotely aggregates the aggregation result to the second data in the next neighbor accelerator card of the accelerator card.

2. The data aggregation method according to claim 1, characterized in that: The number of the accelerator cards is more than 2, and the local accelerator card remotely aggregates first data acquired from the local accelerator card to second data in a next neighbor accelerator card of the local accelerator card, including: In the first working sequence, the accelerator card obtains first data from the accelerator card. In the second working sequence, the acceleration card remotely aggregates the acquired first data to the second data in the next neighboring acceleration card of the acceleration card.

3. The data aggregation method according to claim 1, characterized in that: The local accelerator card performs local aggregation on third data from a previous neighbor accelerator card of the local accelerator card and first data obtained from the local accelerator card, and remotely aggregates the aggregation result to second data in a next neighbor accelerator card of the local accelerator card, including: In the first working sequence, the accelerator card obtains the third data from the accelerator card that is the previous neighbor of the accelerator card. In the second working sequence, the accelerator card obtains the first data from the accelerator card. In the third working sequence, the accelerator card performs a local aggregation operation on the first data and the third data to obtain an aggregation result. In the fourth working sequence, the acceleration card remotely aggregates the aggregation result to the second data in the next neighboring acceleration card of the acceleration card.

4. The data aggregation method according to claim 2, characterized in that: The method further comprises: The accelerator card aggregates the third data from the accelerator card of the previous neighbor of the accelerator card with the fourth data in the accelerator card so that the fourth data is remotely aggregated with the third data, The fifth data in the next neighbor accelerator card is acquired by the next neighbor accelerator card and remotely aggregated to the sixth data in the next neighbor accelerator card of the next neighbor accelerator card, The seventh data of the upper neighbor accelerator card is remotely aggregated with the eighth data from the upper neighbor accelerator card of the upper neighbor accelerator card.

5. The data aggregation method according to claim 4, characterized in that: The first working sequence also includes: In the first working sequence, the previous neighbor accelerator card of the current accelerator card obtains the third data in the previous neighbor accelerator card, and the next neighbor accelerator card obtains the fifth data in the next neighbor accelerator card; The second working sequence also includes: In the second working sequence, the current accelerator card aggregates the third data from the previous neighbor accelerator card of the current accelerator card with the fourth data in the current accelerator card, the next neighbor accelerator card remotely aggregates the acquired fifth data to the sixth data in the next neighbor accelerator card of the next neighbor accelerator card, and the seventh data of the previous neighbor accelerator card is remotely aggregated with the eighth data from the previous neighbor accelerator card of the previous neighbor accelerator card.

6. The data aggregation method according to claim 2, characterized in that: The first working sequence further includes: In the first working sequence, all the acceleration cards in the ring topology network respectively obtain the first data from their local locations; The second working sequence also includes: In the second working sequence, all the acceleration cards in the ring topology network remotely aggregate the first data they obtain to the second data in the next neighboring acceleration card.

7. The data aggregation method according to claim 3, characterized in that: The number of the acceleration cards is 4 or more; The previous neighbor accelerator card aggregates the aggregation result from the previous neighbor accelerator card of the previous neighbor accelerator card with the seventh data in the previous neighbor accelerator card so that the seventh data is remotely aggregated by the aggregation result, and the fifth data in the next neighbor accelerator card is acquired by the next neighbor accelerator card of the next neighbor accelerator card; in, When the number of accelerator cards is 4, the previous neighbor accelerator card of the previous neighbor accelerator card and the next neighbor accelerator card of the next neighbor accelerator card are the same accelerator card.

8. The data aggregation method according to claim 7, characterized in that: The first working sequence also includes: In the first working sequence, the fifth data in the next neighbor accelerator card is acquired by the next neighbor accelerator card of the next neighbor accelerator card, and the previous neighbor accelerator card of the previous neighbor accelerator card acquires the ninth data of the previous neighbor accelerator card; The second working sequence also includes: In the second working sequence, the next neighbor accelerator card of the next neighbor accelerator card obtains the sixth data locally, and the previous neighbor accelerator card of the previous neighbor accelerator card obtains the eighth data locally; The third working sequence also includes: In the third working sequence, the previous neighbor accelerator card of the previous neighbor accelerator card performs a local aggregation operation on the eighth data and the ninth data, and the next neighbor accelerator card of the next neighbor accelerator card performs a local aggregation operation on the fifth data and the sixth data; The fourth working sequence also includes: In the fourth working sequence, the next neighbor accelerator card of the next neighbor accelerator card remotely aggregates the aggregation result to its next neighbor accelerator card, and the previous neighbor accelerator card of the previous neighbor accelerator card remotely aggregates the aggregation result to the previous neighbor accelerator card.

9. The data aggregation method according to claim 3, characterized in that: The first working sequence further includes: In the first working sequence, all the accelerator cards in the ring topology network respectively obtain the third data from their upper neighbor accelerator card; The second working sequence also includes: In the second working sequence, all the acceleration cards in the ring topology network respectively obtain the first data from their local locations; The third working sequence also includes: In the third working sequence, all the accelerator cards in the ring topology network respectively perform local aggregation operations on the first data and the third data. The fourth working sequence also includes: In the fourth working sequence, all the acceleration cards in the ring topology network respectively remotely aggregate the aggregation result to the second data in the next neighboring acceleration card of the local acceleration card.

10. The data aggregation method according to claim 1, characterized in that: The accelerator card is a single-node multi-accelerator card or a multi-node multi-accelerator card. The remote aggregation is performed as follows: The arithmetic control unit in the L2 cache of the accelerator card reads the local data in the L2 cache, reads the remote data from the neighboring accelerator card in the L2 cache, performs operations on the read local data and remote data, obtains an aggregated result, and writes it into the cache location where the local data is located; The local aggregation is performed as follows: The computing core of the accelerator card reads the local data in the cache, reads the remote data from the neighboring accelerator card in the cache, operates on the read local data and remote data, obtains the aggregated result and transmits it.

11. A data aggregation device based on a ring topology network, characterized in that: Multiple accelerator cards are connected into a ring topology network. Any local acceleration card in the ring topology network is used to remotely aggregate first data obtained from the local acceleration card to second data in a next neighboring acceleration card of the local acceleration card, so as to perform one aggregation; or, Any local accelerator card in the ring topology network locally aggregates the third data from the previous neighbor accelerator card of the local accelerator card and the first data obtained from the local accelerator card, and remotely aggregates the second data in the next neighbor accelerator card of the local accelerator card to perform one aggregation; in, The remote aggregation is used to accelerate the arithmetic control unit in the secondary cache of the card to perform operations on the data in the secondary cache. The local aggregation is used to accelerate the computing core in the card to operate on the data in the cache, wherein the cache includes at least one of an input cache, a secondary cache, and a high-bandwidth cache; Any local accelerator card in the ring topology network remotely aggregates first data acquired from the local accelerator card to second data in a next neighbor accelerator card of the local accelerator card, including: The accelerator card obtains first data from the accelerator card, The accelerator card remotely aggregates the acquired first data into second data in a next neighbor accelerator card of the accelerator card; Any local accelerator card in the ring topology network locally aggregates third data from a previous neighbor accelerator card of the local accelerator card with first data obtained from the local accelerator card, and remotely aggregates the aggregation result to second data in a next neighbor accelerator card of the local accelerator card, including: The accelerator card obtains the third data from the accelerator card that is the previous neighbor of the accelerator card. The accelerator card obtains first data from the accelerator card, The accelerator card performs a local aggregation operation on the first data and the third data to obtain an aggregation result, The accelerator card remotely aggregates the aggregation result to the second data in the next neighbor accelerator card of the accelerator card.

Images