Method and Device for Setting Double Ring Path for Efficiently Applying of All Reduce Algorithm in 2D Mesh Structure

Abstract

The present disclosure provides a double ring path setting method and apparatus that can improve learning efficiency by reducing the size of data to be transmitted for each path and reducing communication bottlenecks caused by mutual collective communication between multiple computation nodes by using an edge switching circuit that connects between edge nodes connected to edge computation nodes of a computation array composed of multiple computation nodes in a two-dimensional mesh topology, thereby setting two bidirectional ring paths passing through all computation nodes.

Description

Technology Field

[0001] The disclosed embodiments relate to a double ring path setting method and apparatus, and more specifically, to a double ring path setting method and apparatus capable of applying a double ring all-reduce technique by utilizing edge switching nodes in a two-dimensional mesh structure. Background Technology

[0002] Deep Neural Network (DNN) models are being widely deployed in various application fields, such as image recognition, language modeling, autonomous vehicles, and audio synthesis, due to their remarkable performance improvements, and the focus is on enhancing their accuracy. A common strategy for improving accuracy is to increase the model size, that is, the number of parameters, and as a result, the size and complexity of neural network models continue to grow. Driven by the trend to use large-scale deep neural network models for high accuracy, the number of parameters to be learned has gradually increased, and the size of the datasets required to train these parameters has also grown significantly. To train large deep neural network models, methods utilizing parallel and distributed learning are widely employed.

[0003] In parallel distributed learning, a neural network model is trained using N computation nodes, each performing neural network operations individually. At this time, the parameters of the neural network model to be trained are pre-replicated and transferred to each of the N computation nodes. In other words, the same neural network model is pre-configured. Then, the entire training dataset collected in advance is divided into N equal parts according to the number of computation nodes (N) to create N batches, which are data partitions. Each of the N computation nodes is responsible for the batch assigned to it among the N batches to perform training. Each computation node utilizes its assigned batch to perform neural network operations on the independently configured neural network model, thereby generating local gradients to update the currently set parameters.

[0004] However, the local gradients generated at each of the N computation nodes are gradients obtained from a single batch assigned to each node; since these values ​​are acquired by training on only a subset of the entire training dataset, their reliability is low. Therefore, to train a neural network model using the entire training dataset, it is necessary to obtain a global gradient by integrating multiple local gradients obtained from each computation node. Generally, the global gradient is obtained as the average of multiple local gradients, and the N computation nodes update the parameters of the configured neural network model using the integrated global gradient. Then, by repeatedly performing the process of generating local gradients for the batches reassigned with the updated neural network model parameters and acquiring the global gradient, the neural network model is made to achieve a performance level similar to that of a model trained on the entire dataset. This method of generating a global gradient by integrating local gradients obtained from multiple computation nodes, and then sharing the generated global gradient among multiple computation nodes to synchronize the configured neural network model, is called the All Reduce technique.

[0005] Meanwhile, to use the All Reduce technique, one must either provide a separate synchronization server that receives local gradients from multiple computation nodes to obtain global gradients and distribute them back to the nodes, or have multiple computation nodes perform collective communication to obtain global gradients individually. Furthermore, as the number of computation nodes increases, providing a synchronization server induces massive traffic on the server. On the other hand, if collective communication is performed, the total volume of communication exceeds the network bandwidth, causing communication bottlenecks. Such traffic or bottlenecks act as significant factors that reduce learning efficiency. The problem to be solved

[0006] The objective of the disclosed embodiments is to provide a doubling ring path setting method and apparatus that can improve learning efficiency by reducing communication bottlenecks caused by mutual collective communication between multiple computation nodes configured with a two-dimensional mesh topology during distributed learning according to an all-reduce algorithm for a neural network model.

[0007] The objective of the disclosed embodiments is to provide a double ring path setting method and apparatus that can reduce communication bottlenecks by reducing the size of data to be transmitted for each path by using an edge switching circuit to set two bidirectional ring paths passing through all computation nodes. means of solving the problem

[0008] A double ring path setting method according to an embodiment is a method for applying a double ring all-reduce technique to a multi-chip module comprising a computation array including a plurality of computation nodes connected in a two-dimensional mesh structure, a plurality of edge nodes arranged in four outer directions of the computation array and each connected to a computation node arranged at the outermost of the plurality of computation nodes, and a plurality of edge switch circuits connecting the plurality of edge nodes to each other, the method comprising: a step of setting an initial computation node, an initial main direction, and an initial sub-direction for each of the two ring paths from the plurality of computation nodes, and selecting an adjacent node according to the set initial main direction; a step of selecting an adjacent node connected to the currently set main direction if the selected node is a computation node, and if the adjacent node selected from the computation node is an edge node, reversing the set main direction and selecting another edge node located in the currently set sub-direction; and a step of determining whether the selected computation node is the initial computation node, and if it is the initial computation node, setting two ring paths that sequentially pass through the selected computation node and edge node according to the selected order, and connecting the plurality of edge nodes to each other according to the set ring paths.

[0009] The step of selecting another edge node mentioned above can select an edge node at a distance of 2 hops if there is an edge node at a distance of 2 hops in the sub-direction set from the selected edge node.

[0010] In the step of selecting another edge node, if there is no edge node 2 hops away from the selected edge node in the sub-direction set therein, a neighboring edge node may be selected.

[0011] In the step of selecting another edge node, if there are no neighbor nodes in the sub-direction set from the selected edge node, the currently set sub-direction can be reversed to select an edge node at a distance of 2 hops.

[0012] The two ring paths above may consist of a vertical ring path connecting the plurality of operation nodes in a vertical direction and a horizontal ring path connecting the plurality of operation nodes in a horizontal direction.

[0013] The above vertical ring path may have the main direction set as an upward or downward direction and the sub direction set as a left or right direction, and the above vertical ring path may have the main direction set as a left or right direction and the sub direction set as an upward or downward direction.

[0014] The above-mentioned plurality of edge switch circuits may include a plurality of 4-port switch circuits that connect four edge nodes that intersect each other to select an edge node at a distance of 2 hops, and a plurality of 2-port switch circuits that select neighboring edge nodes.

[0015] The above plurality of edge switch circuits may include a vertical edge switch circuit that connects a plurality of edge nodes located on the upper side of the operation array and a plurality of edge nodes located on the lower side, and a horizontal edge switch circuit that connects a plurality of edge nodes located on the left side of the operation array and a plurality of edge nodes located on the right side.

[0016] The step of connecting the plurality of edge nodes to each other can control the vertical edge switch circuit so that the plurality of edge nodes positioned on the upper side of the operation array are connected to each other and the plurality of edge nodes positioned on the lower side are connected to each other according to the order selected in the vertical ring path that connects the plurality of operation nodes to each other in the vertical direction among the two ring paths.

[0017] The step of connecting the plurality of edge nodes to each other can control the horizontal edge switch circuit so that the plurality of edge nodes placed on the left side of the operation array are connected to each other and the plurality of edge nodes placed on the right side are connected to each other according to the order selected in the horizontal ring path that connects the plurality of operation nodes to each other in a horizontal direction among the two ring paths.

[0018] A double ring path setting device according to an embodiment is a device for setting a double ring path for applying a double ring all-reduce technique to a multi-chip module comprising a computation array including a plurality of computation nodes connected in a two-dimensional mesh structure, a plurality of edge nodes arranged in four outer directions of the computation array and each connected to the computation node arranged at the outermost among the plurality of computation nodes, and a plurality of edge switch circuits connecting the plurality of edge nodes to each other, the device comprising: a memory; and a controller that executes at least a portion of an operation according to a program stored in the memory, wherein the controller sets an initial computation node, an initial main direction, and an initial sub-direction for each of two ring paths from the plurality of computation nodes, and selects an adjacent node according to the set initial main direction; and if the selected node is a computation node, selects an adjacent node connected to the currently set main direction, and if the adjacent node selected from the computation node is an edge node, reverses the set main direction and selects another edge node located in the currently set sub-direction. and determine whether the selected operation node is the initial operation node, and if it is the initial operation node, set two ring paths that sequentially pass through the selected operation node and edge nodes in the selected order, and perform the step of connecting multiple edge nodes to each other according to the set ring paths. Effects of the invention

[0019] Accordingly, the double ring path setting method and device according to the embodiment can set two bidirectional ring paths passing through all computation nodes by using an edge switching circuit that connects edge nodes connected to edge computation nodes of a computation array composed of multiple computation nodes in a two-dimensional mesh topology. Therefore, by reducing the size of data that must be transmitted for each path and thereby reducing communication bottlenecks caused by mutual collective communication between multiple computation nodes, learning efficiency can be improved. Brief explanation of the drawing

[0020] Figures 1 to 3 are diagrams illustrating the concept of the ring-all reduce technique. Figure 4 shows an example of a multi-chip module configuration for a deep learning network accelerator. Figure 5 shows an example of a bidirectional ring all-reduce technique applied to the multi-chip module of Figure 4. FIG. 6 shows a doubling path set to apply a doubling all-reduce technique according to one embodiment to a multi-chip module. FIGS. 7 to 14 are drawings for explaining the process of setting the double ring path of FIG. 6. Figure 15 shows another example of a doubling path set up to apply the doubling all-reduce technique to a multi-chip module. FIG. 16 illustrates a method for setting a doubling path to apply a doubling all-reduce technique according to one embodiment. FIG. 17 is a diagram illustrating a computing environment including a multi-chip module-based accelerator to which a doubling ring all-reduce algorithm according to one embodiment is applied. Specific details for implementing the invention

[0021] Hereinafter, specific embodiments according to the embodiments of the present disclosure will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, apparatuses, and / or systems described herein. However, this is merely illustrative and the present invention is not limited thereto.

[0022] In describing the embodiments of the present disclosure, detailed descriptions of known technology related to the present invention are omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the embodiments. Furthermore, terms described below are defined with consideration of their functions in the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe specific embodiments and should not be limiting. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as “include” or “compose” are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described. Additionally, terms such as “...part,” “...unit,” “module,” and “block” described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.

[0023] As mentioned above, existing All Reduce techniques can degrade learning efficiency by causing excessive traffic on synchronization servers or bottlenecks resulting from collective communication among multiple computation nodes. Accordingly, the Ring All Reduce algorithm has been proposed to efficiently apply the All Reduce technique. In the Ring All Reduce algorithm, N computation nodes are logically configured in a ring topology, and each computation node divides the generated local gradient into N parts, which is the number of computation nodes, to create N chunks of equal size. Then, each of the N computation nodes simultaneously transmits one of the generated N chunks to an adjacent computation node in a designated direction to receive the transmitted chunk. Here, not only is the interval between nodes referred to as a hop, but the process of N computation nodes transmitting and receiving chunks to and from neighboring computation nodes is also referred to as a hop. Each computation node updates the chunk received from the previous hop by summing it with the existing chunk, and then transmits the updated chunk to an adjacent computation node at the next hop. When N computation nodes transmit chunks over N-1 hops and perform an update by summing them, each computation node accumulates the sum results for each of the N chunks, and combining these results yields the sum of N local gradients. Each computation node can obtain the global gradient by calculating the average of the sum of the N local gradients obtained by dividing it by N.

[0024] Figures 1 to 3 are diagrams illustrating the concept of the ring-all reduce technique.

[0025] Figures 1 to 3 assume a case where a ring-all-reduce technique is applied to a system including four operation nodes (Node0 to Node3). If the number of operation nodes (Node0 to Node3) connected by a ring topology structure is four, the entire training dataset can also be divided into four batches and distributed to each operation node (Node0 to Node3). Accordingly, each operation node (Node0 to Node3), which has parameters of the same neural network model already set, utilizes the distributed batches to perform neural network operations according to the neural network model, and obtains local gradients to update the currently set parameters as a result of performing the neural network operations.

[0026] In the ring-all-reduce technique, when a local gradient is obtained by each operation node (Node0 to Node3), as illustrated in FIG. 1(a), the obtained local gradient is divided into equal sizes equal to the total number of operation nodes (Node0 to Node3) (N = 4) to generate four chunks ((a0 to d0) to (a3 to d3)). Typically, since local gradients are obtained as a matrix composed of multiple numbers (e.g., floating-point data), the generated chunks can be obtained as submatrices of the matrix-formatted local gradient. For convenience, the four chunks divided from each local gradient are classified as the a to d chunks, and the chunks obtained from each operation node (Node0 to Node3) are described by classifying them into ((a0 to d0), (a1 to d1), (a2 to d2), and (a3 to d3)) according to the identifier of the operation node (Node0 to Node3).

[0027] Therefore, in the initial state, the four operation nodes (Node0 to Node3) each have obtained four chunks ((a0 to d0), (a1 to d1), (a2 to d2), (a3 to d3)) split from one local gradient, as shown in (a) of FIG. 1. And, similar to the all-reduce technique, the ring all-reduce technique also aims for each of the four operation nodes (Node0 to Node3) to obtain the global gradient. As mentioned above, the global gradient is calculated as the average value of the four local gradients. Therefore, the global gradient can be obtained by accumulating the same sequence of chunks among the a to d chunks generated by splitting from each local gradient, dividing this by the total number of local gradients, i.e., the total number of operation nodes, and then recombining them.

[0028] Accordingly, if a number of operation nodes (Node0 to Node3) can obtain a cumulative chunk ((a0+a1+a2+a3), (b0+b1+b2+b3), (c0+c1+c2+c3), (d0+d1+d2+d3)) representing the cumulative sum for all chunks ((a0 to d0), (a1 to d1), (a2 to d2), (a3 to d3)) as shown in FIG. 1 (b), a global gradient can be easily generated.

[0029] The ring-all-reduce technique is a technique that effectively transmits and accumulates chunks ((a0 ~ d0), (a1 ~ d1), (a2 ~ d2), (a3 ~ d3)) stored in multiple operation nodes (Node0 ~ Node3) that store different chunks ((a0 ~ d0), (a1 ~ d1), (a2 ~ d2), (a3 ~ d3)) to other neighboring nodes, as shown in FIG. 1 (a), so that all operation nodes (Node0 ~ Node3) can obtain an accumulated chunk ((a0+a1+a2+a3), (b0+b1+b2+b3), (c0+c1+c2+c3), (d0+d1+d2+d3)) that allows them to obtain a global gradient, as shown in FIG. 1 (b).

[0030] The ring all-reduce technique is largely divided into two stages: the Reduce Scatter stage and the All Gather stage. FIG. 2 is a diagram illustrating the Reduce Scatter stage, and FIG. 3 is a diagram illustrating the All Gather stage. In FIG. 2 and FIG. 3, (a) to (c) represent the hops where each operation node (Node0 to Node3) transmits a chunk and receives the transmitted chunk, and (d) represents the results of the execution of the Reduce Scatter stage and the All Gather stage. As illustrated in FIG. 2 and FIG. 3, the Reduce Scatter stage and the All Gather stage are each configured to perform a number of hops (N-1) corresponding to the number (N) of operation nodes (Node0 to Node3).

[0031] Referring to FIG. 2, at the first hop of the reduce scatter stage, each of the multiple operation nodes (Node0 to Node3) first selects a different chunk from the four acquired chunks ((a0 to d0), (a1 to d1), (a2 to d2), (a3 to d3)) and transmits it to an adjacent operation node in a specified direction. At this time, the initial chunks selected differently by the operation nodes (Node0 to Node3) and the chunk transmission direction may be pre-set or specified by a synchronization server or controller, etc.

[0032] As illustrated in (a), the 0th operation node (Node0) selects the a-th chunk (a0) and transmits it to the 1st operation node (Node1), the 1st operation node (Node1) selects the b-th chunk (b1) and transmits it to the 2nd operation node (Node2), and the 2nd operation node (Node2) selects the c-th chunk (c2) and transmits it to the 3rd operation node (Node3). Then, the 3rd operation node (Node3) selects the d-th chunk (d3) and transmits it to the 1st operation node (Node1). Since multiple operation nodes (Node0 to Node3) are logically connected in a ring structure, each operation node (Node0 to Node3) can simultaneously transmit and receive chunks (a0, b1, c2, d3) in the same direction.

[0033] When each operation node (Node0 ~ Node3) receives a chunk (a0, b1, c2, d3) from a neighboring operation node, it updates the received chunk (a0, b1, c2, d3) by summing it with a chunk of the same order (a1, b2, c3, d0) among the stored chunks, and stores the updated chunk (a0+a1, b1+b2, c2+c3, d0+d3). Then, each operation node (Node0 ~ Node3) passes the updated chunk (a0+a1, b1+b2, c2+c3, d0+d3) to the next hop, and the neighboring operation node repeats the process of receiving the passed chunk and summing it with a chunk of the same order. At this time, since the updated chunk (a0+a1, b1+b2, c2+c3, d0+d3) is the summed result, the size of the data transmitted to neighboring operation nodes is the same as the size of a single chunk. In other words, data traffic is not increased.

[0034] As shown in FIG. 2 (a) to (c), when the number of operation nodes (Node0 to Node3) is 4 (N = 4), if 3 hops (N-1) are performed, as shown in (d), among the 4 operation nodes (Node0 to Node3), the 0th operation node (Node0) obtains an accumulated chunk in which 4 b-chunks (b0+b1+b2+b3) generated from different operation nodes are summed, the 1st operation node (Node1) obtains an accumulated chunk in which 4 c-chunks (c0+c1+c2+c3) are summed, and the 2nd operation node (Node2) obtains an accumulated chunk in which 4 d-chunks (d0+d1+d2+d3) are summed. And the 3rd operation node (Node3) obtains an accumulated chunk in which 4 a-chunks (a0+a1+a2+a3) are summed. That is, in each of the four operation nodes (Node0 ~ Node3), an accumulation chunk ((a0+a1+a2+a3), (b0+b1+b2+b3), (c0+c1+c2+c3), (d0+d1+d2+d3)) is obtained and stored in a different order.

[0035] Afterwards, at the first hop of the All Gather stage, each operation node (Node0 ~ Node3) passes the accumulated chunk obtained in the Reduce Scatter stage to a neighboring operation node. Then, the process of passing the passed accumulated chunk to a neighboring node is repeated at the next hop. If the process of passing the accumulated chunk is performed for 3 hops (N-1) depending on the number of operation nodes (Node0 ~ Node3) in the All Gather stage, all operation nodes (Node0 ~ Node3) can equally obtain 4 accumulated chunks ((a0+a1+a2+a3), (b0+b1+b2+b3), (c0+c1+c2+c3), (d0+d1+d2+d3)), as shown in (d). That is, as shown in (b) of Fig. 1, cumulative chunks ((a0+a1+a2+a3), (b0+b1+b2+b3), (c0+c1+c2+c3), (d0+d1+d2+d3)) for obtaining global gradients are obtained.

[0036] In this way, when using the ring-all-reduce technique, each operation node (Node0 ~ Node3) not only transmits data in chunks with a size in which the local gradient is divided by the number of operation nodes (N), but also transmits chunks to different neighboring nodes, thereby preventing bottlenecks caused by the transmission of large amounts of data.

[0037] Meanwhile, there is also a growing demand for hardware-implemented neural network accelerators to accommodate deep neural network models that have increased in size and complexity. However, as the size and complexity of models increase, the size of the chips constituting the accelerators also gradually increase, leading to a problem of reduced yield. To address this issue, research on neural network accelerators based on Multi-chip Modules (MCMs)—which replace the functions of a large chip by being composed of multiple small chiplets rather than being implemented as a single large chip—is being actively conducted recently.

[0038] Figure 4 shows an example of a multi-chip module configuration for a deep learning network accelerator, and Figure 5 shows an example of a bidirectional ring-all-reduce technique applied to the multi-chip module of Figure 4.

[0039] Referring to FIG. 4, the MCM (10) includes a computation array (11) composed of a plurality of computation nodes (Nd00 to Nd15) arranged in a two-dimensional mesh structure, and a plurality of edge nodes (12) connected to an edge computation node placed at the outermost edge among the plurality of computation nodes (Nd00 to Nd15) arranged in the computation array (11).

[0040] Here, multiple computation nodes (Nd00 to Nd15) are each implemented as chiplets and can perform neural network computations independently. In contrast, multiple edge nodes (12) are configured to allow multiple computation nodes (Nd00 to Nd15) of the computation array (11) to transmit or receive data to an external controller or memory, and can be implemented as interface pads (PADs), etc. As described above, the MCM (10) can be utilized in a neural network accelerator, and the operation of multiple computation nodes (Nd00 to Nd15) can be controlled by a controller included with the MCM (10) in the neural network accelerator or by a processor of a computing device equipped with the accelerator. Furthermore, the bidirectional ring all reduce technique according to one embodiment and the double ring path setting method for applying it can also be performed under the control of the controller or processor. Therefore, the controller or processor controlling the operation of the computation nodes (Nd00 to Nd15) can also be referred to as a double ring path setting device.

[0041] Multiple operation nodes (Nd00 to Nd15) are connected by a logical 2D mesh structure, allowing data to be transmitted or received from neighboring operation nodes or edge nodes located in the vertical or horizontal direction. Here, the vertical and horizontal directions do not refer to physical directions but are expressions representing the logical connection relationships between operation nodes (Nd00 to Nd15) connected by the 2D mesh structure. Additionally, each operation node (Nd00 to Nd15) can transmit and receive data bidirectionally with adjacent neighboring operation nodes or edge nodes. For example, in the case of the edge operation node, the 00 operation node (Nd00), it is connected not only to the 01 and 04 operation nodes (Nd01, Nd04) located on the right and lower sides, but also to the edge node (12) located on the upper and left sides, and can transmit data to the connected 01 and 04 operation nodes (Nd01, Nd04) and the edge node (12) located on the upper and left sides, and can also receive data from the 01 and 04 operation nodes (Nd01, Nd04) and the edge node (12) located on the upper and left sides. However, multiple edge nodes (12) are each connected only to adjacent edge operation nodes.

[0042] When applying the ring-all-reduce technique in an MCM (10) in which multiple operation nodes (Nd00 to Nd15) are connected in a 2D mesh structure as shown in FIG. 5, a Hamiltonian path that can be circulated through all operation nodes (Nd00 to Nd15) only once is searched, and the searched Hamiltonian path is set as the ring path for the ring-all-reduce technique. Then, according to the ring path set at each hop, each operation node (Nd00 to Nd15) is enabled to transmit chunks to neighboring operation nodes.

[0043] However, since multiple operation nodes (Nd00 to Nd15) can transmit and receive data in both directions, unlike FIGS. 1 to 3, where chunks can only be transmitted in one direction of the ring path, chunks can be transmitted in both directions along a single ring path. Therefore, in an MCM (10) capable of bidirectional transmission, local gradients can be divided into chunks (2N) that are twice the number of operation nodes (N, where N = 16), and N of the 2N chunks divided into twice the number can be transmitted in opposite directions. At this time, since local gradients of the same size are divided into 2N rather than N, the data size of each chunk becomes half the size of the chunks shown in FIGS. 1 to 3.

[0044] In this way, even when applying the ring-all-reduce technique to divide local gradients into 2N chunks and transmit them in different directions N at a time, the reduce-scatter step and the all-gather step each need to perform N-1 hops, so 2(N-1) hops must be performed, just like the ring-all-reduce technique of FIGS. 1 to 3. However, since the chunk size is reduced by half and communication efficiency increases by twofold, bottlenecks can be prevented more effectively.

[0045] FIG. 6 shows a doubling path set to apply a doubling all-reduce technique according to one embodiment to a multi-chip module, and FIGS. 7 to 14 are drawings for explaining the process of setting the doubling path of FIG. 6.

[0046] In FIG. 6, the MCM (10) also includes a computation array (11) composed of a plurality of computation nodes (Nd00 to Nd15) arranged in a two-dimensional mesh structure, as in FIG. 4, and a plurality of edge nodes (12) connected to the edge computation node placed at the outermost among the plurality of computation nodes (Nd00 to Nd15) arranged in the computation array (11). It further includes edge switch circuits (21 to 24) that electrically connect the plurality of edge nodes to each other.

[0047] As mentioned above, the bidirectional ring-all-reduce technique can more effectively reduce bottlenecks because the size of the chunk transmitted by each operation node (Nd00 to Nd15) is reduced by half compared to the unidirectional ring-all-reduce technique. In addition, each of the multiple operation nodes (Nd00 to Nd15) connected by a 2D mesh structure can perform bidirectional transmission with neighboring operation nodes or edge nodes not only in the vertical direction but also in the horizontal direction. That is, each operation node (Nd00 to Nd15) can perform bidirectional transmission in four directions.

[0048] Nevertheless, in the bidirectional ring all-reduce technique illustrated in FIG. 5, since the ring path is formed such that each operation node (Nd00 to Nd15) performs bidirectional transmission in only two of the four directions, only half of the total available bandwidth can be used at each hop, making it very inefficient. However, it is impossible to search for two Hamiltonian paths that do not overlap with each other while passing through all operation nodes (Nd00 to Nd15) once within the operation array (11). This is because the ring path passing through some of the operation nodes (Nd00 to Nd15) is set as a path in both the vertical and horizontal directions. For example, in the operation array (11) shown in FIG. 5, the ring path is set by changing direction at right angles at the 00, 03, 05, 07, 09, 11, 12 and 15 operation nodes (Nd00, Nd03, Nd05, Nd07, Nd09, Nd11, Nd12, Nd15), so both horizontal and vertical paths are utilized.

[0049] The newly proposed double ring all reduce technique here extends the bidirectional ring all reduce technique to establish two non-overlapping ring paths and enables bidirectional transmission in each of the two ring paths where chunks are set.

[0050] However, when attempting to set up a ring path using only multiple operation nodes (Nd00 to Nd15), as described above, it is not possible to set up two ring paths that do not overlap each other. To overcome this limitation, the present invention further includes first to fourth edge switch circuits (21 to 24) capable of controlling the connection between multiple edge nodes. Each edge switch circuit (21 to 24) can be controlled by a controller included in the accelerator containing the MCM (10) or by a processor outside the accelerator.

[0051] The first to fourth edge switch circuits (21 to 24) are each configured to connect edge nodes (12) placed in one of the four directions of the operation array (11). For example, the first and second edge switch circuits (21, 22) may each connect edge nodes placed on the upper and lower sides of the operation array (11), and the third and fourth edge switch circuits (23, 24) may each connect edge nodes placed on the right and left sides of the operation array (11).

[0052] In this way, when multiple edge switch circuits (21 to 24) connect edge nodes to each other so that chunks transmitted from one edge operation node can be delivered to another edge operation node, as illustrated in FIG. 6, one of the two ring paths can be configured to pass through all operation nodes (Nd00 to Nd15) only in the vertical direction, while the other ring path can be configured to pass through all operation nodes (Nd00 to Nd15) only in the horizontal direction. That is, two ring paths can be configured to pass through all operation nodes (Nd00 to Nd15) without the paths overlapping. This is because the change of direction from vertical to horizontal or from horizontal to vertical is made at the edge node that is previously used simply as an interface. Here, the ring path configured to pass through the operation nodes (Nd00 to Nd15) in the vertical direction can be distinguished as a vertical ring path, and the ring path configured to pass through them in the horizontal direction can be distinguished as a horizontal ring path. And the first and second edge switch circuits (21, 22) arranged on the upper and lower sides of the operation array (11) and used to form a vertical ring path are called vertical edge switch circuits, and the third and fourth edge switch circuits (23, 24) arranged on both sides of the operation array (11) and used to form a horizontal ring path are called horizontal edge switch circuits.

[0053] When two bidirectional ring paths are established to allow different chunks to be transmitted through the two bidirectional ring paths, the size of the chunk transmitted by each operation node (Nd00 to Nd15) to a neighboring operation node is reduced to one-fourth compared to the unidirectional ring-all-reduce technique. Therefore, chunks are transmitted with the same number of hops (2(N-1)) as in the ring-all-reduce technique, but the size of the transmitted chunk can be reduced to one-fourth of the chunk size according to the unidirectional ring-all-reduce technique illustrated in FIGS. 1 to 3, thereby allowing for more effective reduction of bottlenecks. In other words, efficient learning can be performed even when the data size of the local gradients according to the size of the neural network model to be learned is very large.

[0054] Each edge switch circuit (21 to 24) can connect edge nodes located in one of the four directions of the computation array (11) in various combinations. For example, as shown in FIG. 6, the first and third edge switch circuits (21, 23) can be configured to connect all edge nodes located in the upper direction and the right direction to each other. However, if each edge switch circuit (21 to 24) is designed to arbitrarily connect all edge nodes placed in a specific direction, as the number of computation nodes included in the computation array (11) increases, the structure of the edge switch circuits (21 to 24) becomes very complex, and the routing algorithm for control also becomes very complex, which causes an increase in cost. In addition, if the distance between edge nodes connected to each other by the edge switch circuits (21 to 24) is significantly different, a large time difference occurs in the transmission of chunks between the computation nodes, which may reduce transmission efficiency. For example, the ring path illustrated in FIG. 5 may be extended to an edge node and configured to be connected by edge switch circuits (21 to 24), but in this case, the transmission distance between the 00th operation node (Nd00) and the 12th operation node (Nd12) becomes relatively very long.

[0055] To resolve this problem, two ring paths are set in FIG. 6 so that the transmission distance between operation nodes can be made as uniform as possible. FIGS. 7 to 14 illustrate a method for setting a vertical ring path among the two ring paths shown in FIG. 6. Referring to FIGS. 7 to 14, the method for setting a vertical ring path first involves selecting one operation node among a plurality of operation nodes (here, for example, the 00th operation node (Nd00)) as the initial operation node, as shown in FIG. 7.

[0056] Next, the initial direction for transmitting the chunk is set. At this time, both a main direction and a sub-direction can be set as the initial direction. Since a vertical ring path is being established here, either an upward or downward direction can be selected as the initial main direction. Along with this, the horizontal direction to be switched at the first reached edge node can be set as the initial sub-direction. The sub-direction can be selected as either a left or a right direction. On the other hand, when establishing a horizontal ring path, either the left or the right direction is initially selected as the initial main direction, and subsequently, either an upward or downward direction can be selected as the initial sub-direction.

[0057] When an edge node is selected by the initial direction set, the main direction is reversed, and it is checked whether an edge node located at a distance of 2 hops from the connected edge node, rather than an edge node adjacent to the selected sub-direction, can be selected. If an edge node at a distance of 2 hops can be selected, the edge node at a distance of 2 hops is selected. Then, as shown in FIG. 8, the operation node (Nd02) connected to the main direction reversed from the selected edge node is selected, and operation nodes (Nd06, Nd10, Nd14) connected sequentially in the vertical direction are selected. Then, when an edge node connected to the edge operation node (Nd14) is selected, the main direction is reversed again, and it is determined whether an edge node at a distance of 2 hops can be selected again from the currently set sub-direction. If an edge node at a distance of 2 hops cannot be selected, a neighboring edge node is selected as shown in FIG. 9. The edge operation node (Nd15) connected to the neighboring edge node is selected. Then, as shown in FIG. 10, operation nodes (Nd11, Nd07, Nd03) are sequentially selected from the edge operation node (Nd15) connected to the selected edge node according to the currently set main direction. Afterwards, it is determined whether an edge node at a distance of 2 hops is selected from the edge node (12) selected by the selected edge operation node (Nd03) in the currently set sub-direction, and if an edge node at a distance of 2 hops cannot be selected, it is determined whether a neighbor edge node can be selected in the currently selected horizontal direction. If neither the 2-hop distance nor the neighbor edge node can be selected, the direction of the sub-direction is reversed to determine again whether a 2-hop distance edge node can be selected, and if it can be selected, a 2-hop distance edge node is selected in the reversed direction as shown in FIG. 11. Then, as shown in FIG. 12, operation nodes (Nd05, Nd09, Nd13) connected to the main direction are sequentially selected from the edge operation node (Nd01) connected to the selected edge node.As before, when an edge node connected to the edge operation node (Nd13) is selected again, since an edge node at a distance of 2 hops from the currently selected sub-direction cannot be selected, an edge node at a distance of 1 hop is selected as in FIG. 13, and as in FIG. 14, operation nodes (Nd08, Nd04, Nd00) connected to the main direction from the edge operation node (Nd12) connected to the selected edge node are sequentially selected. Here, since the selected operation node (Nd00) is the initial operation node set at the beginning, multiple operation nodes (Nd00 ~ Nd15) are connected in a ring structure. Therefore, the path traversed during the process up to this point is set as a vertical ring path.

[0058] Although FIGS. 7 to 14 illustrate only the process of setting the vertical ring path among the two ring paths, the method of setting the horizontal ring path can be performed in the same way, with only the main direction and sub direction being alternated between the horizontal and vertical directions. At this time, the same operation node may be selected as the initial operation node in the vertical ring path and the horizontal ring path, but different initial operation nodes may also be selected.

[0059] When two ring paths are each configured in the manner described above, the maximum path distance is the path connecting from an edge computation node to another edge computation node via an edge node that is 2 hops away, passing through an edge node connected to the edge computation node. Therefore, compared to cases where ring paths are configured in a different manner, the difference in path distance between computation nodes can be set to a minimum.

[0060] When a ring path is established in this manner, the edge switch circuits (21 to 24) may be configured to include a 4-port switch circuit capable of connecting four consecutively arranged edge nodes and a 2-port switch circuit capable of connecting two adjacent edge nodes. That is, the switch circuits may be configured as 4-port or 2-port switch circuits, allowing for low-cost implementation and control. Here, the 2-port switch circuit is utilized only to connect neighboring edge nodes in the established ring path, while the 4-port switch circuit is utilized for the remainder.

[0061] Figure 15 shows another example of a doubling path set up to apply the doubling all-reduce technique to a multi-chip module.

[0062] In FIGS. 4 to 14, an MCM (10) having an operation array (11) with 16 operation nodes (Nd00 to Nd15) arranged in a 4 x 4 array is illustrated, but in FIG. 15, an MCM (20) having an operation array (21) with 64 operation nodes arranged in an 8 x 8 array is illustrated. In FIG. 15, the ring path method described in FIG. 7 to 14 can be applied as is, and two ring paths can be established. In this case as well, the edge switch circuits (41 to 44) can be configured as 4-port or 2-port switch circuits.

[0063] Since the case where the ring path being configured needs to connect neighboring edge nodes occurs at the outer edge of the operation array (11, 21), the 2-port switch circuit is also used to connect the two edge nodes located at the outermost edge among the multiple edge nodes listed in each direction. Accordingly, in FIGS. 6 and FIGS. 15, the 2-port switch circuit is configured to connect two edge nodes located at both ends among the multiple edge nodes arranged in the lower direction and the multiple edge nodes arranged in the left direction, but it can be adjusted in various ways.

[0064] In the illustrated embodiments, each component may have different functions and capabilities in addition to those described above and may include additional components not described. Additionally, in one embodiment, each component may be implemented using one or more physically separated devices, or by one or more processors or a combination of one or more processors and software, and may not be clearly distinguished in specific operation as in the illustrated examples.

[0065] And the accelerator having the MCM (10, 20) illustrated in FIGS. 6 and 15 may be implemented in a logic circuit by hardware, firmware, software, or a combination thereof, and may also be implemented using a general-purpose or specific-purpose computer. The device may be implemented using a hardwired device, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc. Additionally, the device may be implemented as a system-on-chip (SoC) including one or more processors and controllers.

[0066] Furthermore, the accelerator may be installed in a computing device or server equipped with hardware elements in the form of software, hardware, or a combination thereof. A computing device or server may refer to various devices that include, in whole or in part, communication devices such as communication modems for communicating with various devices or wired / wireless communication networks, memory for storing data for executing programs, and microprocessors for executing programs to perform calculations and commands.

[0067] FIG. 16 illustrates a method for setting a doubling path to apply a doubling all-reduce technique according to one embodiment.

[0068] Referring to FIG. 16, the double ring path setting method sets an initial operation node in an operation array in which a plurality of operation nodes are arranged (51). Here, the initial operation node can be set for each of the vertical ring path and the horizontal ring path, and the same operation node may be selected and set for each path. Along with this, an initial direction is set for each of the vertical and horizontal ring paths (52). As described above, the double ring path consists of two paths, a vertical ring path and a horizontal ring path, and a main direction and a sub-direction can be set for each path. In the vertical ring path, either the upward direction or the downward direction is first set as the initial main direction, and then either the left or the rear direction can be selected as the initial sub-direction. In the horizontal ring path, either the left or the rear direction is first selected as the initial main direction, and either the upward direction or the downward direction can be set as the initial sub-direction.

[0069] When the initial direction is set, a neighbor node is selected according to the currently set main direction from the selected initial operation node, and it is determined whether the selected node is an edge node located outside the operation array (53). If the node selected according to the initial main direction, which is the currently set main direction, is an edge node, the currently set main direction is reversed (54). Then, it is determined whether an edge node at a distance of 2 hops from the selected edge node can be selected in the currently set sub-direction (55). If an edge node at a distance of 2 hops exists in the sub-direction and can be selected, the edge node at a distance of 2 hops is selected (56). However, if there is no edge node at a distance of 2 hops in the currently set sub-direction, it is determined whether a neighbor edge node can be selected (57). If it is determined that a neighbor edge node can be selected, the neighbor edge node is selected (58). When a 2-hop distance or neighbor edge node is selected, the operation node connected to the selected edge node is selected (59). At this time, the operation node connected to the edge node is an edge operation node located in the currently set main direction. However, if not only the 2-hop distance edge node but also the neighbor edge node cannot be selected, the currently set sub-direction is reversed (60). Then, it is determined whether the 2-hop edge node can be selected again in the reversed sub-direction (55).

[0070] That is, multiple operation nodes of the operation array (11) have paths set only in the main direction, and path switching from the main direction to the sub direction or from the sub direction to the main direction is performed only at the edge nodes.

[0071] Meanwhile, if it is determined that the main direction of the selected operation node is not the edge node direction, a neighbor operation node located adjacent to the current main direction is selected (61).

[0072] When an operation node is selected, it is determined whether the selected operation node is an initial operation node (62). That is, it is determined whether the path formed by the sequentially selected nodes has returned to the initial position, which is the initial operation node. If the initial operation node is not selected, it is determined again whether the selected node is an edge node according to the currently set main direction from the selected operation node (53). However, if it is determined that the initial operation node is selected and a ring path has been formed, the multiple edge switch circuits are controlled so that the edge nodes are connected to each other along the ring path, thereby setting it as a vertical ring path or a horizontal ring path (63).

[0073] Afterwards, learning efficiency is improved by applying a double ring all-reduce algorithm that simultaneously performs a bidirectional ring all-reduce algorithm using both of the two ring paths, the vertical ring path and the horizontal ring path.

[0074] The double ring path setting method illustrated in Fig. 16 can be performed by a controller included in an accelerator equipped with an MCM or by an external processor.

[0075] Although FIG. 16 describes each process as being executed sequentially, this is merely an illustrative description, and a person skilled in the art can apply various modifications and variations by changing the order described in FIG. 16, executing one or more processes in parallel, or adding other processes, as long as they do not deviate from the essential characteristics of the embodiment of the present invention.

[0076] FIG. 17 is a diagram illustrating a computing environment including a multi-chip module-based accelerator to which a doubling ring all-reduce algorithm according to one embodiment is applied.

[0077] In the illustrated embodiments, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below. The illustrated computing environment (90) may include a computing device (91) to perform the doubling path setting method illustrated in FIG. 16. In one embodiment, the computing device (91) may be one or more components included in the doubling path setting device illustrated in FIG. 6.

[0078] A computing device (91) includes at least one processor (92), a computer-readable storage medium (93), a communication bus (95), and an accelerator (97). The accelerator may include a controller (98) and an MCM (99). The MCM (99) may include a plurality of computation nodes, a plurality of edge nodes, and a plurality of edge switch circuits for interconnecting the plurality of edge nodes. The controller (98) controls the plurality of edge switch circuits and the plurality of computation nodes.

[0079] A processor (92) or controller (98) may establish a doubling path for a plurality of computation nodes and a plurality of edge nodes of an MCM (99) according to the exemplary embodiment described above, and may perform a proposed doubling all-reduce algorithm according to the established doubling path. At this time, the processor (92) or controller (98) may execute one or more programs (94) stored in a computer-readable storage medium (93). The one or more programs (94) may include one or more computer-executable instructions, and the computer-executable instructions may be configured to cause a computing device (91) to perform operations according to the exemplary embodiment when executed by the processor (92).

[0080] The communication bus (95) interconnects various other components of the computing device (91), including the processor (92), computer-readable storage medium (93), and accelerator (97).

[0081] The computing device (91) may also include one or more communication interfaces (96).

[0082] Although the present invention has been described in detail above through representative embodiments, those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the technical spirit of the appended claims.

Claims

Claim 1 A method for setting a double ring path for applying a double ring all-reduce technique to a multi-chip module comprising: a computation array including a plurality of computation nodes connected by a two-dimensional mesh structure; a plurality of edge nodes arranged in four outer directions of the computation array and each connected to the computation node arranged at the outermost of the plurality of computation nodes; and a plurality of edge switch circuits connecting the plurality of edge nodes to each other, the method comprising: a step of setting an initial computation node, an initial main direction, and an initial sub-direction for each of the two ring paths from the plurality of computation nodes, and selecting an adjacent node according to the set initial main direction; a step of selecting an adjacent node connected to the currently set main direction if the selected node is a computation node, and if the adjacent node selected from the computation node is an edge node, reversing the set main direction and selecting another edge node located in the currently set sub-direction; and a step of determining whether the selected computation node is the initial computation node, and if it is the initial computation node, setting two ring paths that sequentially pass through the selected computation node and the edge node according to the selected order, and connecting the plurality of edge nodes to each other according to the set ring paths. Claim 2 In claim 1, the step of selecting the other edge node is a double ring path setting method in which, if there is an edge node at a distance of 2 hops in a sub-direction set from the selected edge node, the edge node at a distance of 2 hops is selected. Claim 3 In claim 1, the step of selecting another edge node is a double ring path setting method in which, if there is no edge node at a distance of 2 hops in a sub-direction set from the selected edge node, a neighbor edge node is selected. Claim 4 In claim 1, the step of selecting another edge node is a double ring path setting method in which, if there is no neighbor node in the sub-direction set from the selected edge node, the currently set sub-direction is reversed to select an edge node at a distance of 2 hops. Claim 5 A double ring path setting method according to claim 1, wherein the two ring paths consist of a vertical ring path connecting the plurality of operation nodes in a vertical direction and a horizontal ring path connecting the plurality of operation nodes in a horizontal direction. Claim 6 A double ring path setting method according to claim 5, wherein the main direction of the vertical ring path is set as an upward or downward direction and the sub direction is set as a left or right direction, and the main direction of the vertical ring path is set as a left or right direction and the sub direction is set as an upward or downward direction. Claim 7 A double ring path setting method according to claim 1, wherein the plurality of edge switch circuits comprises a plurality of 4-port switch circuits that connect four edge nodes that intersect each other to select an edge node at a distance of 2 hops, and a plurality of 2-port switch circuits that select neighboring edge nodes. Claim 8 A double ring path setting method according to claim 1, wherein the plurality of edge switch circuits include a vertical edge switch circuit that connects a plurality of edge nodes positioned on the upper side of the operation array and a plurality of edge nodes positioned on the lower side, and a horizontal edge switch circuit that connects a plurality of edge nodes positioned on the left side of the operation array and a plurality of edge nodes positioned on the right side. Claim 9 In claim 8, the step of connecting the plurality of edge nodes to each other is a double ring path setting method that controls the vertical edge switch circuit so that the plurality of edge nodes placed on the upper side of the operation array are connected to each other and the plurality of edge nodes placed on the lower side are connected to each other according to the order selected in the vertical ring path that connects the plurality of operation nodes to each other in a vertical direction among the two ring paths. Claim 10 In claim 8, the step of connecting the plurality of edge nodes to each other is a double ring path setting method that controls the horizontal edge switch circuit such that, according to a selected order in a horizontal ring path connecting the plurality of operation nodes in a horizontal direction among the two ring paths, the plurality of edge nodes positioned on the left side of the operation array are connected to each other, and the plurality of edge nodes positioned on the right side are connected to each other. Claim 11 In any one of claims 1 to 10, the double ring path setting method is a double ring path setting method performed by the processor of a device comprising a memory and a processor. Claim 12 A device for setting a double ring path for applying a double ring all-reduce technique to a multi-chip module comprising a computation array including a plurality of computation nodes connected by a two-dimensional mesh structure, a plurality of edge nodes arranged in four outer directions of the computation array and each connected to the computation node arranged at the outermost of the plurality of computation nodes, and a plurality of edge switch circuits connecting the plurality of edge nodes to each other, the device comprising: a memory; and a controller that executes at least a portion of an operation according to a program stored in the memory, wherein the controller sets an initial computation node, an initial main direction, and an initial sub-direction for each of two ring paths from the plurality of computation nodes, and selects an adjacent node according to the set initial main direction; and if the selected node is a computation node, selects an adjacent node connected to the currently set main direction, and if the adjacent node selected from the computation node is an edge node, reverses the set main direction and selects another edge node located in the currently set sub-direction. A double ring path setting device that determines whether a selected operation node is the initial operation node, and if it is the initial operation node, sets two ring paths that sequentially pass through the selected operation node and edge nodes in the selected order, and performs the step of connecting a plurality of edge nodes to each other according to the set ring paths. Claim 13 In claim 12, the controller is a double ring path setting device that selects an edge node at a distance of 2 hops in a sub-direction set from an edge node selected as a neighbor node of a computation node if such an edge node exists, and selects a neighbor edge node if no edge node at a distance of 2 hops exists. Claim 14 In claim 12, the controller is a double ring path setting device that, if there is no neighbor node in the sub-direction set at the edge node selected as a neighbor node of the operation node, reverses the currently set sub-direction to select an edge node at a distance of 2 hops. Claim 15 In claim 12, the controller is a double ring path setting device that sets two ring paths: a vertical ring path connecting the plurality of operation nodes in a vertical direction and a horizontal ring path connecting the plurality of operation nodes in a horizontal direction. Claim 16 In claim 15, the controller sets the main direction of the vertical ring path to an upward or downward direction and sets the sub direction to a left or right direction, and sets the main direction of the vertical ring path to a left or right direction and sets the sub direction to an upward or downward direction, a double ring path setting device. Claim 17 In claim 12, the double ring path setting device comprises a plurality of edge switch circuits that intersect each other to select an edge node at a distance of 2 hops, a plurality of 4-port switch circuits that connect each other to select an edge node at a distance of 2 hops, and a plurality of 2-port switch circuits that select neighboring edge nodes. Claim 18 A double ring path setting device according to claim 12, wherein the plurality of edge switch circuits comprises a vertical edge switch circuit that connects a plurality of edge nodes positioned on the upper side of the operation array and a plurality of edge nodes positioned on the lower side, and a horizontal edge switch circuit that connects a plurality of edge nodes positioned on the left side of the operation array and a plurality of edge nodes positioned on the right side. Claim 19 In claim 18, the controller is a double ring path setting device that controls the vertical edge switch circuit such that, according to a selected order in a vertical ring path connecting the plurality of operation nodes in a vertical direction among the two ring paths, a plurality of edge nodes placed on the upper side of the operation array are connected to each other, and a plurality of edge nodes placed on the lower side are connected to each other. Claim 20 In claim 18, the controller is a double ring path setting device that controls the horizontal edge switch circuit such that, according to a selected order in a horizontal ring path connecting the plurality of operation nodes in a horizontal direction among the two ring paths, the plurality of edge nodes placed on the left side of the operation array are connected to each other, and the plurality of edge nodes placed on the right side are connected to each other.

Images