Computer system

Abstract

To achieve group communications with low latency on the basis of identification information received from a lower-level node without providing a management device that manages the group communications.SOLUTION: A computer system includes a plurality of first nodes connected in a tree shape, each having a calculation function, and at least one second node connected to a leaf side of the first node. A second participation node as the second node that participates in group communications transmits target data for the group communications and identification information that identifies the node that participates in the group communications to the first node. Each of first participation nodes as the first node that participates in the group communications calculates the target data received from the second participation node, and transmits at least a part of identification information received from the second participation node or identification information that identifies the node that participates in the group communications generated based on at least a part of the identification information received from the second participation node together with the calculation result.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to computer systems. [Background technology]

[0002] Communication systems that solve large-scale problems by coordinating multiple accelerators connected via a network through collective communication are widely used. In addition, to realize collective communication with low latency, a computer system is known in which relay devices such as network switches have a computing function and the accelerators directly communicate with each other without going through a host processor.

[0003] This type of computer system has, for example, a collective communication management device that sets in advance device allocation information, etc., in the table of each relay device for each combination (group) of devices participating in collective communication. Each relay device then refers to the allocation information set in correspondence with the identification information that identifies the group received from the lower layer, and executes collective communication according to the referenced allocation information. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a system for performing collective communication using identification information that identifies a node received from a subordinate node. [Means for solving the problem]

[0005] A computer system according to an embodiment of the present invention is a computer system having a plurality of first nodes connected in a tree configuration, each having a calculation function, and at least one second node connected to the leaf side of the first nodes, wherein a second participating node which is a second node participating in a collective communication transmits target data for the collective communication and identification information identifying the node participating in the collective communication to the first node, and each of the first participating nodes which is a first node participating in the collective communication calculates the target data received from the second participating node and transmits identification information identifying the node participating in the collective communication which is generated based on at least a portion of the identification information received from the second participating node or at least a portion of the identification information received from the second participating node, together with the calculation result, to a higher node. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram illustrating an example of a computer system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an explanatory diagram showing an example of collective communication executed by the computer system of FIG. [Diagram 3] 2 is a flow diagram showing an example of the operation of each network switch in FIG. 1. [Figure 4] FIG. 11 is an explanatory diagram showing an example of collective communication executed by another computer system. [Diagram 5] FIG. 11 is an explanatory diagram showing an example of collective communication by a computer system according to another embodiment of the present invention. [Figure 6] FIG. 11 is an explanatory diagram showing an example of collective communication by a computer system according to another embodiment of the present invention. [Figure 7] FIG. 11 is an explanatory diagram showing an example of collective communication by a computer system according to another embodiment of the present invention. [Figure 8] FIG. 11 is an explanatory diagram showing an example of collective communication by a computer system according to another embodiment of the present invention. [Figure 9] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an accelerator or a network switch in the above-described embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0008] Fig. 1 is a block diagram showing an example of a computer system according to an embodiment of the present invention. The computer system SYS1 shown in Fig. 1 has a plurality of network switches SW (SW21-SW23, SW31) and a plurality of accelerators AC (AC11-AC19) connected to the network switches SW21-SW23 located at leaves as viewed from SW31. Each of the accelerators AC (AC11-AC19) is located at a leaf as viewed from the network switches SW21-SW23 to which it is connected. In this way, the network switches SW and the accelerators AC are connected in a tree shape. For example, the network switch SW has an arithmetic unit used for data reduction.

[0009] The accelerator AC and the network switch SW are examples of nodes. The network switch SW is an example of a first node, and the accelerator AC is an example of a second node. The network switch SW31 is an example of a root node located at the root, and the network switches SW21-SW23 are examples of leaf nodes. The accelerator AC may be called a leaf node. Hereinafter, the network switch SW is also simply called a switch SW.

[0010] The two-digit numbers added to the accelerator AC and switch SW are identification numbers indicating the accelerator AC and switch SW, with the upper digits indicating the tree hierarchy and the lower digits indicating the identification number within the hierarchy. The identification number is an example of identification information for identifying a node. The identification information for identifying a node is information used to identify a node participating in collective communication, and may be information that corresponds one-to-one with the node and is different for each node. Since each accelerator AC and each switch SW recognizes the identification number of its own node, it can recognize whether the received identification number is for its own node, a node lower than its own node, or a node higher than its own node. The identification number may be an address assigned to each node.

[0011] The switch SW21 is connected to the lower accelerators AC11, AC12, AC13 and the upper switch SW31. The switch SW22 is connected to the lower accelerators AC14, AC15, AC16 and the upper switch SW31. The switch SW23 is connected to the lower accelerators AC17, AC18, AC19 and the upper switch SW31. The connections between the accelerators AC and the switches SW and between the switches SW are made via cables (signal lines) such as electric cables or optical cables. The computer system SYS1 is capable of performing collective communication such as contraction and aggregation.

[0012] Each switch SW21-SW23, SW31 has a storage unit such as a register REG that stores the identification number of a node lower than the node itself among a plurality of identification numbers received from a lower node. Each switch SW21-SW23, SW31 can specify the lower node indicated by the stored identification number as a participating node of the collective communication by referring to the identification number stored in the storage unit, and can transmit the contracted data to the lower node specified as a participating node of the collective communication without recognizing in advance the tree structure of the switches SW used in the collective communication. Note that the computer system may be configured so that each node transmits and receives the identification numbers of all nodes participating in the collective communication in each communication between the nodes. In this computer system, each node may recognize the lower node that is the source of the data to be contracted from the identification number received together with the data from the lower node, wait for the reception of a packet from the recognized node, recognize the lower node that is the destination of the data that is the contracted result from the identification number received together with the data from the higher node, and transmit the data that is the contracted result to the recognized node. In this case, each node may not have the above-mentioned storage unit.

[0013] Furthermore, each of the switches SW21-SW23, SW31 has a function of relaying packets and a function of calculating data included in the payload of the packets. Each of the switches SW21-SW23, SW31, for example, calculates data received from a lower node according to a command code included in the packet header. For example, the types of calculations include sum, product, maximum value, minimum value, etc. In the following, an example in which the computer system SYS1 executes a contraction operation is described, but the computer system SYS1 may execute collective communication other than a contraction operation (for example, an aggregation operation).

[0014] Each of the switches SW21-SW23 executes an operation on the data included in the packet according to the packet received from the accelerator AC participating in the collective communication, and transmits a packet including the data obtained by the operation to the switch SW31.

[0015] The switch SW31 executes an operation on the data included in the packet according to the packet received from each of SW21-SW23. The switch SW31 transmits a packet including the data obtained by the operation to each of the switches SW21-SW23. Each of the switches SW21, SW22, and SW23 transmits the data included in the packet received from the switch SW31 as a packet to the lower accelerator AC participating in the collective communication. Then, the contraction process, which is an example of collective communication, is completed.

[0016] Note that information transmission between the accelerator AC and the switch SW, and information transmission between the switches SW, may be performed using a method other than packets. Furthermore, the computer system SYS1 may have a tree structure with four or more hierarchical levels. The number of ports on the leaf side of each switch SW may be two or four or more. This allows collective communication to be performed using a tree of any shape made up of any number of nodes among the multiple nodes (multiple accelerators AC and multiple switches SW) mounted on the computer system SYS1.

[0017] Fig. 2 is an explanatory diagram showing an example of collective communication executed by the computer system SYS1 of Fig. 1. Fig. 2 shows an example of a control method of the computer system SYS1. For example, the computer system SYS1 executes contraction of data of group 1 and contraction of data of group 2. Fig. 2 shows the flow of contraction. In group 1 and group 2, the nodes participating in the collective communication and the target data of the collective communication are different from each other.

[0018] The accelerator AC and switch SW shown in a solid line frame, and the transmission path shown with a solid arrow connected to the solid line frame, are used for collective communication. The direction of the solid arrow indicates the packet transfer direction at the time of contraction. The number added to the solid arrow indicates the identification number of the node participating in the collective communication, and is stored, for example, in the packet header. The accelerator AC and switch SW shown in a dashed line frame, and the dashed transmission path connected to the dashed line frame, are not used for collective communication. The switch SW shown in a solid line frame is an example of a first participating node participating in collective communication. The accelerator AC shown in a solid line frame is an example of a second participating node participating in collective communication.

[0019] An accelerator AC participating in collective communication transmits at least one of the identification numbers of the nodes participating in the collective communication to a higher-level switch SW. For example, at least one of the accelerators AC in a group of accelerators AC participating in collective communication connected to each of the switches SW21-SW23 transmits the identification number of an accelerator AC participating in collective communication in its own group to the switch SW. Also, at least one of the accelerators AC in a group of accelerators AC participating in collective communication connected to each of the switches SW21-SW23 transmits the identification number of a switch SW participating in collective communication to the switch SW. However, the identification number of the switch SW transmitted by the accelerator AC does not include the identification number of the switch SW31, which is the root node, among the identification numbers of the switches SW participating in collective communication.

[0020] The identification number and the target data (payload) of the collective communication transmitted by the accelerator AC are transmitted to the accelerator AC participating in the collective communication from a host processor (not shown) connected to the computer system SYS1. In other words, the accelerator AC participating in the collective communication is the accelerator AC that has received the identification number and data from the host processor.

[0021] The switch SW that receives a packet from the accelerator AC or a switch SW at a lower level than the node itself compresses data contained in the packet received from a node lower than the node itself among the nodes indicated by the identification numbers contained in the packet. The switch SW stores in the packet header at least the identification numbers that need to be transmitted to the higher level node among the identification numbers contained in the packet received from the lower level node. At this time, the switch SW may store in the packet header the identification numbers contained in the packet received from the lower level node as they are, or may store in the packet header the identification numbers contained in the packet received from the lower level node excluding the identification number used to identify the lower level node. The switch SW then places (stores) the compressed data in the payload to generate a packet, and transmits the generated packet to the switch SW at the next higher level.

[0022] In this way, each switch SW can recognize the node from which the data to be condensed is sent and the node to which the condensed result is to be sent, based on the multiple identification numbers received, without having to recognize in advance the tree structure of the switches SW used for collective communication.

[0023] If the received packet does not contain the identification number of its own node, the switch SW recognizes that its own node is the root node. The switch SW of the root node compresses the data contained in the received packet and transfers the compressed data to the lower node. The compressed data by the root node is then transferred to the accelerator AC, and the compression process is completed.

[0024] Each switch SW can recognize that it is the root node based on the received identification number without having to recognize in advance the tree structure of the switches SW used in the collective communication. Therefore, for example, even when switches SW of any tree structure are flexibly combined according to the scale of the collective communication without providing a collective communication management device that manages the switches SW participating in the collective communication, any switch SW can be operated as the root node.

[0025] For example, in collective communication for group 1, accelerators AC11 and AC12 connected to switch SW21 are included in one group, and accelerator AC15 connected to switch SW22 is included in another group. The group of accelerators AC11 and AC12 transmits identification numbers 11 and 12 of accelerators AC11 and AC12 in its own group, and identification numbers 21 and 22 of switches SW21 and SW22 participating in the collective communication, excluding switch SW31, the root node. Note that the node transmitting the identification numbers 11, 12, 21, and 22 may be either accelerator AC11 or AC12.

[0026] The group of accelerator AC15 transmits the identification number 15 of the accelerator AC15 of its own group and the identification numbers 21, 22 of the switches SW21, SW22 participating in the collective communication excluding the switch SW31 of the root node. This makes it possible to execute a desired collective communication without setting in advance in each switch SW the allocation information of the node that executes the collective communication.

[0027] The switch SW21 receives a packet including identification numbers 11 and 12, an instruction code, and data from the accelerator AC11, and receives a packet including identification numbers 21 and 22, an instruction code, and data from the accelerator AC12. Of the received identification numbers 11, 12, 21, and 22, the switch SW21 stores the identification numbers 11 and 12 of the next lower hierarchical level than its own node in the register REG. Note that a storage unit such as a buffer or a memory may be provided instead of the register REG.

[0028] The switch SW21 compresses data received from the accelerators AC11 and AC12 according to the command code included in the packet header. The command code may be included in at least one of the packet headers received from the accelerators AC11 and AC12. The switch SW21 stores the identification numbers 21 and 22 and the command code in the packet header, stores the compressed data in the payload to generate a packet, and transmits the generated packet to the switch SW31.

[0029] The switch SW22 receives a packet including identification numbers 15, 21, and 22 and data from the accelerator AC15. Of the received identification numbers 15, 21, and 22, the switch SW22 stores the identification number 15, which is one layer below its own node, in the register REG. The switch SW22 compresses the data received from the accelerator AC15 according to the instruction code included in the packet header. The switch SW22 stores the identification numbers 21 and 22 and the instruction code in the packet header, stores the compressed data in the payload to generate a packet, and transmits the generated packet to the switch SW31.

[0030] The switch SW31, which is the root node, receives a packet including identification numbers 21, 22, an instruction code, and data from the switch SW21, and receives a packet including identification numbers 21, 22, an instruction code, and data from the switch SW22. Of the received identification numbers 21, 22, the switch SW31 stores the identification numbers 21, 22 of the next lower hierarchical level than the own node in the register REG. The switch SW31 that does not receive a packet including an identification number of a hierarchical level equal to or higher than the own node is the root node. Therefore, the switch SW31 compresses the data included in the received packet, and transmits the packet including the compressed data to the nodes (SW21, SW22) indicated by the identification numbers 21, 22 stored in the register REG.

[0031] Switch SW21, which receives a packet from switch SW31, creates and transmits a packet including the data contained in the packet from switch SW31 to the nodes (AC11, AC12) indicated by identification numbers 11 and 12 stored in register REG. Switch SW22, which receives a packet from switch SW31, creates and transmits a packet including the data contained in the packet from switch SW31 to the node (AC15) indicated by identification number 15 stored in register REG. Then, the contraction operation of group 1 by computer system SYS1 is completed.

[0032] In collective communication for group 2, the same operation as the contract operation for group 1 is executed. A detailed description of the operation similar to the contract operation for group 1 is omitted.

[0033] In collective communication for group 2, switch SW22 receives identification numbers 14 and 15, an instruction code, and data from accelerator AC14, and receives identification numbers 22 and 23, an instruction code, and data from accelerator AC15. Switch SW22 stores identification numbers 14 and 15 in register REG and contracts the received data according to the instruction code. Switch SW22 transmits identification numbers 22 and 23, the instruction code, and the contracted data to switch SW31.

[0034] The switch SW23 receives the identification numbers 18 and 19, the instruction code, and data from the accelerator AC18, and receives the identification numbers 22 and 23, the instruction code, and data from the accelerator AC19. The switch SW23 stores the identification numbers 18 and 19 in the register REG and compresses the received data according to the instruction code. The switch SW23 transmits the identification numbers 22 and 23, the instruction code, and the compressed data to the switch SW31.

[0035] Switch SW31 receives identification numbers 22 and 23, an instruction code, and data from switch SW23, and receives identification numbers 22 and 23, an instruction code, and data from switch SW23. Switch SW31 stores identification numbers 21 and 22 in register REG, and contracts the received data according to the instruction code. Switch SW31 then transmits the contracted data to the nodes (SW22, SW23) indicated by the identification numbers 21 and 22 stored in register REG. Thereafter, the contraction results are transferred sequentially to nodes in lower hierarchical levels, and the contraction operation of group 2 by computer system SYS1 is completed.

[0036] Fig. 3 is a flow diagram showing an example of the operation of each network switch SW in Fig. 1. For example, each network switch SW starts the operation shown in Fig. 3 in response to a start-up instruction from the computer system SYS1. Fig. 3 shows an example of a control method for the computer system SYS1. Note that the identification number of a participating node in collective communication is an example of identification information for identifying a participating node in collective communication.

[0037] First, in step S10, the switch SW waits for a packet from a leaf node. Next, in step S12, if the switch SW receives a packet, it executes step S14, and if the switch SW does not receive a packet, it returns to step S10.

[0038] In step S14, the switch SW acquires the identification number of the participating node of the collective communication from the information stored in the packet header. For example, the identification number of the participating node of the collective communication includes the identification number of the accelerator AC that transmits the packet including the data to be compressed and the identification number of the switch SW (excluding the root node) that executes the compression.

[0039] Next, in step S16, the switch SW judges whether or not it has acquired the identification numbers of all the participating nodes of the collective communication (excluding the root node). If the switch SW has acquired the identification numbers of all the participating nodes of the collective communication, it executes step S18, and if there are any participating node identification numbers that it has not acquired, it returns to step S10. Note that whether or not the switch SW has acquired the identification numbers of all the participating nodes of the collective communication may be judged by whether or not a predetermined time has elapsed since the reception of the first packet.

[0040] In addition, when the system SYS1 is configured so that each node identifies the identification numbers of nodes in the hierarchy immediately below its own node from the identification numbers stored in the packet headers received from its own node's immediately lower nodes, and waits to receive packets from all of the immediately lower nodes corresponding to the identified identification numbers, the switch SW proceeds with the processing of steps S10-S16 as follows.

[0041] First, the switch SW waits for a packet from a leaf node, i.e., a node in a hierarchy immediately below itself, through a loop of steps S10 and S12 (NO), and when it receives a packet from a node in a hierarchy immediately below itself (S12: YES), it executes step S14. In step S14, the switch SW obtains and stores an identification number equal to or greater than 1 stored in the header of the received packet.

[0042] Next, the switch SW repeats steps S10-S14 until it receives a packet header including the identification number of the node in the immediately lower hierarchical level (immediately lower node identification number) as determined in step S16. For example, if the switch SW has already received a packet header including the immediately lower node identification number, it repeats steps S10-S14 until it receives packets from all immediately lower nodes corresponding to the immediately lower node identification numbers recognized by the reception (S16: NO).

[0043] When the switch SW receives packets from all the directly below nodes corresponding to the directly below node identification number recognized by the reception, it determines that all the identification numbers received from the directly below nodes up to that point and stored in the processing of step S14 have been acquired as the identification numbers of the participating nodes in the collective communication (S16: YES), and the processing proceeds to step S18.

[0044] In step S18, the switch SW executes a reduction operation on the data included in the received packet, for example, by using a built-in calculator. The calculator executes the reduction operation according to the instruction code stored in the packet header. Each switch SW may have multiple types of calculators according to the types of instruction codes.

[0045] Next, in step S20, the switch SW judges whether or not the identification numbers of the nodes participating in the collective communication contained in the received packet include the identification number of a node at a level higher than the node itself. That is, the switch SW judges whether or not the node itself is a root node. If the packet contains the identification number of a node at a level higher than the node itself, the switch SW is not a root node and therefore executes step S22. If the packet does not contain the identification number of a node at a level higher than the node itself, the switch SW is a root node and therefore executes step S30.

[0046] In step S22, the switch SW transmits a packet including the identification number of the participating node in the collective communication and the data obtained by the contraction operation to the root side node. Next, in step S24, the switch SW waits for a packet from the root side node. Next, in step S26, if the switch SW receives a packet from the root side node, it executes step S28, and if it does not receive a packet from the root side node, it returns to step S24. In step S28, the switch SW transmits the data (contracted data) included in the packet received from the root side node to the leaf side node identified as the node participating in the collective communication, and returns the operation to step S10.

[0047] In step S30, the switch SW transmits the data obtained by the reduction operation in step S18 to the leaf side nodes identified as nodes participating in the collective communication, and returns the operation to step S10.

[0048] As described above, in this embodiment, each switch SW can recognize the node that is the sender of the data to be compressed and the node that is the destination of the compressed result based on the received multiple identification numbers, without having to recognize in advance the tree structure of the switch SW used for collective communication. This allows the computer system SYS1 to realize flexible collective communication using any tree structure, without having to set in advance in each switch SW the allocation information of the node that executes collective communication.

[0049] In other words, the computer system SYS1 can execute collective communication using any switch SW without providing a collective communication management device that sets allocation information and the like in advance in a table or the like for each switch SW. That is, each switch SW does not need to have a table or the like in which node information for each combination of switches SW used in collective communication is set in advance. Therefore, each switch SW can recognize the participating nodes of collective communication based on multiple identification numbers received from lower nodes without searching a table or the like, and can execute data contraction operations. As a result, the computer system SYS1 can realize collective communication with low latency compared to the case where the participating nodes of collective communication are recognized by searching a table or the like.

[0050] Fig. 4 is an explanatory diagram showing an example of collective communication executed by another computer system. Detailed explanations of elements similar to those in Fig. 1 and Fig. 2 are omitted. In the computer system SYS0 shown in Fig. 4, the tree structure of the accelerator AC and the switch SW is the same as that of the computer system SYS1 in Fig. 1.

[0051] However, the computer system SYS0 is assigned a group ID for each group of nodes participating in collective communication. The group ID is identification information that identifies a group, not identification information that identifies the nodes included in the group. The computer system SYS0 also has a collective communication management device that manages a table (dashed line frame) provided in each switch SW. The collective communication management device is connected to each switch SW, and sets information such as an identification number to be used in collective communication for each group ID in the table of each switch SW before collective communication is executed. In other words, the collective communication management device sets in advance the route of the tree to be used for collective communication for each group ID.

[0052] Each accelerator AC transmits a packet including the data to be compressed and a group ID (g1 or g2 in this example) to the switch SW one layer higher in the hierarchy. Each switch SW has a table in which information identifying adjacent nodes participating in the collective communication is set corresponding to the group ID included in the received packet. The identification number following the code dn in the table indicates the node from which the data to be compressed is sent. The identification number following the code up in the table indicates the node to which the compressed data is sent. The underlines in the table indicate the information of the group ID that is the target of the collective communication. The code (none) in the table indicates that the compression operation is not performed.

[0053] Each switch SW refers to information in the table corresponding to the group ID included in the packet, and obtains the identification number of the node from which the data to be reduced is to be reduced and the identification number of the node to which the data after the reduction is to be reduced. Each switch SW then reduces the data received from the lower node based on the information set in the table, and transmits the reduced data to the upper node.

[0054] For example, in collective communication with group ID=g1, switch SW21 refers to the table based on group ID=g1 received from accelerators AC11 and AC12, and obtains identification numbers 11 and 12 of accelerators AC that are the transmission sources of data to be compressed. Switch SW21 also refers to the table and obtains identification number 31 of switch SW that is the transmission destination of the compressed data. Switch SW21 then compresses the data received from accelerators AC11 and AC12, and transmits the compressed data together with group ID=g1 to switch SW31.

[0055] Similarly, the switch SW22 refers to the table based on the group ID=g1 received from the accelerator AC15, compresses the data received from the accelerator AC15, and transmits the compressed data together with the group ID=g1 to the switch SW31. The switch SW31 refers to the table based on the group ID=g1 received from the switches SW21 and SW22, and compresses the data received from the switches SW21 and SW22. Since the code "up" is not written in the table of the switch SW31, the switch SW31 recognizes that its own node is the root node. Then, the switch SW31 transmits the compressed data to the switches SW21 and SW22 indicated by the identification numbers written in the table together with the code "dn".

[0056] The switch SW21, which has received data from the switch SW31, transmits the compressed data to the accelerators AC11 and AC12, which are indicated by the identification numbers described in the table together with the code dn. The switch SW22, which has received data from the switch SW31, transmits the compressed data to the accelerator AC15, which is indicated by the identification number described in the table together with the code dn.

[0057] The collective communication of group ID=g2 is executed in the same manner as the collective communication of group ID=g1. The switch SW22 refers to the table based on the group ID=g2 received from the accelerators AC14 and AC15, compresses the data received from the accelerators AC14 and AC15, and transmits the compressed data together with the group ID=g2 to the switch SW31. The switch SW23 refers to the table based on the group ID=g2 received from the accelerators AC18 and AC19, compresses the data received from the accelerators AC18 and AC19, and transmits the compressed data together with the group ID=g2 to the switch SW31.

[0058] The switch SW31 refers to the table based on the group ID=g2 received from the switches SW22 and SW23, and compresses the data received from the switches SW22 and SW23. Then, the switch SW31 transmits the compressed data to the switches SW22 and SW23 indicated by the identification numbers described in the table together with the code dn.

[0059] The switch SW22, which has received data from the switch SW31, transmits the compressed data to the accelerators AC14 and AC15, which are indicated by the identification numbers described in the table together with the code dn. The switch SW23, which has received data from the switch SW31, transmits the compressed data to the accelerators AC18 and AC19, which are indicated by the identification numbers described in the table together with the code dn.

[0060] In the computer system SYS0 shown in FIG. 4, a collective communication management device is used to set information such as an identification number to be used for collective communication for each group ID in the table of each switch SW in advance. For this reason, the scale of the computer system SYS0 is larger than that of a computer system that does not have a collective communication management device. Also, each time a new group for collective communication is generated, the table of each switch SW must be updated. Furthermore, each switch SW must receive a group ID from a lower node and search the table based on the group ID to recognize the nodes participating in the collective communication. As a result, it is difficult to realize collective communication with low latency.

[0061] Fig. 5 is an explanatory diagram showing an example of collective communication by a computer system in another embodiment of the present invention. The same elements as those in Figs. 1 and 2 are given the same reference numerals, and detailed description will be omitted. The computer system SYS2 shown in Fig. 5 has a plurality of accelerators AC (AC11-AC19, AC1A-AC1C) having a tree structure and a plurality of switches SW (SW21-SW24, SW31-SW34, SW41). Fig. 5 shows an example of a control method for the computer system SYS2 together with the configuration of the computer system SYS2.

[0062] Although not shown, each switch SW has a storage unit such as a register REG, similar to the switch SW in FIG. 1. The accelerator AC and the switch SW are an example of a node. The computer system SYS2 also has a plurality of CPUs (Central Processing Units; CPU1-CPU3). Although the type of cable is not particularly limited, the nodes SW21-SW24, SW31-SW34, and SW41 are connected to other nodes SW via communication cables. The node SW23 may be directly connected to the outside of the computer system SYS2, or may be housed in another server SV (not shown) together with the accelerator AC and CPU.

[0063] Node SW21 is connected to lower side nodes AC11-AC14 and upper side node SW31. Node SW22 is connected to lower side nodes AC15-AC18 and upper side node SW31. Node SW24 is connected to lower side nodes AC19-AC1C and upper side node SW31. Node SW31 is connected to lower side nodes SW21-SW24 and upper side node SW41. Node SW41 is connected to lower side nodes SW31-SW34.

[0064] CPU1 is connected to nodes AC11-AC14. CPU2 is connected to nodes AC15-AC18. CPU3 is connected to nodes AC19-AC1C. Each of CPU1-CPU3 transmits data to be used in collective communication, information indicating nodes (AC, SW) participating in the collective communication, and the like, together with an instruction code and data to be compressed, to node AC in server SV.

[0065] Although not particularly limited, the computer system SYS2 may have a plurality of servers SV (SV1-SV4) each having a CPU, a node AC, or a node SW. In this case, for example, the CPU1, the nodes AC11-AC14, and the node SW21 are mounted on the server SV1. The CPU2, the nodes AC15-AC18, and the node SW22 are mounted on the server SV2. The CPU3, the nodes AC19-AC1C, and the node SW24 are mounted on the server SV3. The nodes SW31-SW34, SW41 are mounted on the server SV4. The server SV4 has a CPU (not shown). The nodes SW32-SW34 may receive data used in the collective communication and information indicating the nodes (SW) participating in the collective communication, together with an instruction code and data to be contracted, from the CPU (not shown).

[0066] For example, in this embodiment, each CPU transmits in parallel to all participating nodes AC of its own server SV the identification numbers of the node AC of its own server SV and all nodes SW (except the root node) participating in the collective communication as information indicating the nodes participating in the collective communication. For example, CPU1 transmits a plurality of identification numbers indicating nodes AC11, AC13, AC14, SW21, SW22, SW24, SW31, SW32 to nodes AC11, AC13, AC14 together with an instruction code and data to be contracted. CPU2 transmits a plurality of identification numbers indicating nodes AC16, AC18, SW21, SW22, SW24, SW31, SW32 to nodes AC16, AC18 together with an instruction code and data to be contracted. CPU3 transmits a plurality of identification numbers indicating nodes AC19, AC1B, SW21, SW22, SW24, SW31, SW32 to nodes AC19, AC1B together with an instruction code and data to be contracted.

[0067] Nodes AC11, AC13, and AC14 participating in the collective communication are an example of a group connected to node SW21. Nodes AC16 and AC18 participating in the collective communication are an example of a group connected to node SW22. Nodes AC19 and AC1B participating in the collective communication are an example of a group connected to node SW24.

[0068] Each node AC participating in the collective communication transmits the identification number, the instruction code, and the data to be compressed received from the CPU to the higher-level node SW (SW21, SW22, or SW24). As shown in Fig. 5, the nodes AC participating in the collective communication connected to each CPU transmit the same identification number in parallel to the corresponding node SW.

[0069] Node SW21 receives the same identification numbers 11, 12, 14, 21, 22, 24, 31, and 32 in parallel from nodes AC11, AC13, and AC14, respectively. Node SW21 stores the identification numbers 11, 13, and 14 of the next lower hierarchical level of its own node in a register REG (not shown). Node SW21 contracts the data received from nodes AC11, AC13, and AC14 in accordance with the instruction code. Then, node SW21 transmits the contracted data and the identification numbers 21, 22, 24, 31, and 32, excluding the identification numbers 11, 13, and 14 used to identify the lower nodes, from among the received identification numbers, to node SW31. Node SW21 may transmit all of the received identification numbers 11, 13, 14, 21, 22, 24, 31, and 32 to node SW31 as they are, without excluding the identification numbers 11, 13, and 14 used to identify the lower nodes.

[0070] The operation of nodes SW22 and SW24 is the same as that of node SW21. Node SW22 compresses the data received from nodes AC16 and AC18, and transmits the compressed data and the identification numbers 21, 22, 24, 31, and 32, excluding the identification numbers 16 and 18, used to identify the lower nodes, to node SW31. Node SW24 compresses the data received from nodes AC19 and AC1B, and transmits the compressed data and the identification numbers 21, 22, 24, 31, and 32, excluding the identification numbers 19 and 1B, used to identify the lower nodes, to node SW31. Note that nodes SW22 and SW24 may transmit all the received identification numbers as they are to node SW31, without excluding the identification numbers used to identify the lower nodes.

[0071] Node SW31 receives the same identification numbers 21, 22, 24, 31, and 32 from nodes SW21, SW22, and SW24, respectively, and the data contracted by each of nodes SW21, SW22, and SW24. Node SW31 stores the identification numbers 21, 22, and 24 of the next lower level of its own node in a register REG (not shown). Node SW31 contracts the data received from nodes SW21, SW22, and SW24, and transmits the contracted data and the identification numbers 31 and 32 excluding the identification numbers 21, 22, and 24 used to identify the lower nodes to node SW41. Node SW31 may transmit all the received identification numbers as they are to node SW41, without excluding the identification numbers used to identify the lower nodes.

[0072] Node SW32 receives the same identification numbers 25, 26, 28, 31, and 32 from lower nodes SW25, SW26, and SW28 (not shown), and data contracted by each of nodes SW25, SW26, and SW28. Node SW32 stores the identification numbers 25, 26, and 28 of the next lower level of its own node in a register REG (not shown). Node SW32 contracts the data received from nodes SW25, SW26, and SW28, and transmits the contracted data and the identification numbers 31 and 32, excluding the identification numbers 25, 26, and 28 used to identify the lower nodes, to node SW41. Node SW32 may transmit all the received identification numbers as they are to node SW41, without excluding the identification numbers used to identify the lower nodes.

[0073] Node SW41 does not receive an identification number from nodes SW31 and SW32 that is higher than its own node's hierarchy level, and therefore recognizes itself as the root node. Node SW41 stores the identification numbers 31 and 32 of the hierarchy level lower than its own node in a register REG (not shown). Node SW41 compresses the data received from nodes SW31 and SW32, and transmits the compressed data to nodes SW31 and SW32 indicated by the identification numbers 31 and 32 stored in the register REG. After this, the lower node SW that received the compressed data transmits the compressed data to the node SW or node AC that is one level lower and is participating in the collective communication, as in the explanation of FIG. 2. Then, the collective communication is completed.

[0074] As described above, this embodiment can also provide the same effects as the above-mentioned embodiment. For example, the computer system SYS2 can realize flexible collective communication using an arbitrary tree structure without setting in advance in each switch SW the allocation information of the nodes that perform collective communication. As a result, the computer system SYS2 can realize collective communication with low latency compared to the case where the computer system SYS2 recognizes the participating nodes of collective communication by searching a table or the like.

[0075] Furthermore, in this embodiment, each node SW receives multiple identical identification numbers in parallel from the lower node AC or the lower node SW. Therefore, each node SW can generate an identification number to be transmitted to the upper node SW using multiple identification numbers received from either the lower node AC or the lower node SW. For example, each node SW can prepare an identification number to be transmitted to the upper node SW before receiving other packets by using the identification number contained in the packet received earliest. This allows collective communication to be realized with even lower latency. Also, each of CPU1-CPU3 can transmit a common identification number to all nodes AC in the group, which simplifies the transmission process of the identification numbers.

[0076] Fig. 6 is an explanatory diagram showing an example of collective communication by a computer system in another embodiment of the present invention. The same elements as those in Figs. 1, 2 and 5 are given the same reference numerals, and detailed explanations are omitted. The hardware configuration and tree structure of the computer system SYS3 shown in Fig. 6 are the same as those of the computer system SYS2 in Fig. 5. Fig. 6 shows an example of a control method for the computer system SYS3 together with the configuration of the computer system SYS3.

[0077] For example, in this embodiment, each CPU (CPU1-CPU3) transmits data to be processed to nodes AC participating in the collective communication in its own server SV. Each CPU transmits multiple identification numbers indicating the participating nodes AC of the collective communication in its own server SV to one of the participating nodes AC in its own server SV. Each CPU transmits the identification numbers of the node SW to the minimum number of participating nodes AC so that the identification numbers of all nodes SW participating in the collective communication are covered by all of CPU1-CPU3.

[0078] Each CPU may transmit to the participating node AC of its own server SV the identification numbers of all participating nodes AC and SW in its own server SV and the identification numbers of the higher participating node SW in approximately the same number without duplication. By transmitting the identification numbers of the nodes AC participating in the collective communication evenly from the CPU to the nodes AC without duplication, it is possible to save the communication bandwidth between the CPU and the nodes AC and between the nodes AC and SW.

[0079] For example, CPU1 transmits identification numbers 11, 13, and 14, an instruction code, and data to be compressed to node AC11, and transmits identification numbers 21, 22, and 24, an instruction code, and data to be compressed to node AC13. CPU1 transmits only the payload to node AC14.

[0080] CPU2 transmits identification numbers 16 and 17, instruction codes, and data to be compressed to node AC16, and transmits identification numbers 31 and 32, instruction codes, and data to be compressed to node AC18. CPU3 transmits identification numbers 19 and 1B, instruction codes, and data to be compressed to node AC19, and transmits identification numbers 21, 22, and 24, instruction codes, and data to be compressed to node AC1B.

[0081] Each of the nodes SW21, SW22, and SW24 operates in the same manner as the nodes SW21, SW22, and SW24 in Fig. 5. For example, the node SW21 compresses data received from the nodes AC11, AC13, and AC14. The node AC14 transmits only the payload (i.e., only the data to be compressed) to the node SW21. The node SW21 transmits to the node SW31 the identification numbers 21, 22, and 24, excluding the identification numbers 11, 13, and 14 used to identify the lower nodes, among the received identification numbers, the instruction code, and the compressed data. The node SW21 may transmit to the node SW31 all the received identification numbers 11, 13, 14, 21, 22, and 24, excluding the identification numbers 11, 13, and 14 used to identify the lower nodes.

[0082] Node SW22 compresses the data received from nodes AC16 and AC18. Node SW22 transmits to node SW31, among the received identification numbers, identification numbers 31 and 32 (excluding identification numbers 16 and 17 used to identify lower nodes), instruction codes, and compressed data. Node SW24 compresses the data received from nodes AC19 and AC1B. Node SW24 transmits to node SW31, among the received identification numbers, identification numbers 21, 22, and 24 (excluding identification numbers 19 and 1B used to identify lower nodes), instruction codes, and compressed data. Node SW24 may transmit to node SW31 all the received identification numbers as they are, without excluding the identification numbers used to identify lower nodes.

[0083] Each of the nodes SW31, SW32, and SW41 operates in the same manner as the nodes SW31, SW32, and SW41 in Fig. 5. However, the node SW32 transmits only the payload to the node SW41. Then, the data compressed by the node SW41 is transferred in sequence to the nodes SW31, SW32, SW21, SW22, SW24, and the participating node AC of the collective communication, and the collective communication is completed.

[0084] As described above, this embodiment can also provide the same effects as the above-mentioned embodiment. For example, the computer system SYS3 can realize flexible collective communication using an arbitrary tree structure without setting in advance in each switch SW the allocation information of the nodes that execute collective communication. As a result, the computer system SYS3 can realize collective communication with low latency compared to the case where the computer system SYS3 recognizes the participating nodes of collective communication by searching a table or the like.

[0085] Furthermore, in this embodiment, each CPU transmits the identification numbers of nodes AC and SW participating in collective communication to node AC almost equally without duplication, which makes it possible to save the communication band between the CPU and node AC, the communication band between nodes AC and SW, and the communication band between nodes SW and SW.

[0086] Fig. 7 is an explanatory diagram showing an example of collective communication by a computer system in another embodiment of the present invention. The same elements as those in Figs. 1, 2 and 5 are given the same reference numerals, and detailed explanations are omitted. The hardware configuration and tree structure of the computer system SYS4 shown in Fig. 7 are the same as those of the computer system SYS2 in Fig. 5. Fig. 7 shows an example of a control method for the computer system SYS4 together with the configuration of the computer system SYS4.

[0087] However, each node AC and each node SW is not assigned a unique identification number, but is assigned a value corresponding to the height of the tree structure. For example, in the tree structure shown in Fig. 7 where the height of switch SW is "3", node AC connected to leaf node SW (SW2) is assigned "1", and node SW connected to node AC is assigned "2". Node SW, which is the root node, is assigned "4", and node SW connected to node SW4 is assigned "3".

[0088] Each CPU transmits, in parallel, to nodes AC participating in the collective communication, height information corresponding to the height of node SW on the tree and port information indicating the port number of node SW connected to the lower node, as identification information indicating node SW participating in the collective communication. For example, identification information "(2, (a, c, d))" indicates that a packet is transmitted to ports a, c, and d of corresponding node SW2. Identification information "(3, (a, b, d))" indicates that a packet is transmitted to ports a, b, and d of corresponding node SW3. Identification information "(4, (a, b))" indicates that a packet is transmitted to ports a and b of corresponding node SW4. In this way, the identification information includes multiple pieces of information (e.g., multiple port IDs) that are different from each other and identify multiple nodes participating in the collective communication.

[0089] For example, node SW2 of server SV1 degenerates data contained in a packet received at ports a, c, d based on receiving identification information "(2, (a, c, d))" and stores the identification information "(2, (a, c, d))" in a register REG or the like (not shown). Furthermore, node SW2 of server SV1 transmits, among the identification information received at ports a, c, d, identification information "(3, (a, b, d))" and identification information "(4, (a, b))" excluding the identification information "(2, (a, c, d))" used to identify the lower node AC1, together with the degenerated data and instruction code to the higher node. Note that this node SW2 may send all of the identification information "(2,(a,c,d))", "(3,(a,b,d))" and "(4,(a,b))" received at ports a, c and d directly to the higher level nodes, without excluding the identification information "(2,(a,c,d))" used to identify the lower level node AC1. Node SW2 of servers SV2 and SV3 operates in the same way as node SW2 of server SV1.

[0090] Node SW3 connected to server SV1 degenerates the data received at ports a, b, and d based on the reception of identification information "(3, (a, b, d))" and stores "(3, (a, b, d))" in a register REG or the like. In addition, node SW3 transmits, among the identification information received at ports a, b, and d, identification information "(4, (a, b))" excluding the identification information "(3, (a, b, d))" used to identify the lower node AC1, together with the degenerated data and instruction code to the upper node. Note that this node SW3 may transmit all of the identification information received at ports a, b, and d to the upper node as is, without excluding the identification information used to identify the lower node AC1.

[0091] Based on receiving the identification information "(4,(a,b))", node SW4 reduces the data received at ports a and b, and stores the identification information "(4,(a,b))" in a register REG or the like. Furthermore, since node SW4 does not receive any identification information indicating a hierarchy above its own node, it recognizes itself as the root node. Therefore, node SW4 transmits the reduced data to lower nodes via ports a and b indicated by the stored identification information "(4,(a,b))". Note that if each node SW has independent input and output ports, node SW4 transmits the reduced data to output ports a and b corresponding to input ports a and b indicated by the identification information "(4,(a,b))".

[0092] Thereafter, the lower node SW that has received the condensed data transmits the condensed data to the participating nodes of the collective communication, among the next lower node SW or node AC, in the same manner as described in FIG. 2. For example, node SW3 connected to servers SV1-SV3 transmits the condensed data transmitted from node SW4 to lower node SW2 via ports a, b, and d indicated by the stored identification information "(3, (a, b, d))". Node SW2 of server SV1 transmits the condensed data transmitted from node SW3 to lower node AC via ports a, c, and d indicated by the stored identification information "(2, (a, c, d))". Then, the collective communication is completed.

[0093] As described above, this embodiment can also provide the same effects as the above-mentioned embodiment. For example, the computer system SYS4 can realize flexible collective communication using an arbitrary tree structure without setting in advance in each switch SW the allocation information of the nodes that execute collective communication. As a result, the computer system SYS4 can realize collective communication with low latency compared to the case where the computer system SYS4 searches a table or the like to recognize the participating nodes of collective communication.

[0094] Furthermore, in this embodiment, collective communication can be performed by mutually transmitting height information of each node SW participating in collective communication and port information of the port in each node SW connected to the lower node SW (or AC) as identification information between the participating nodes in the collective communication.

[0095] In Fig. 7, the group of participating nodes AC connected to each of the leaf nodes SW transmits the same identification information in parallel. However, the group of participating nodes AC connected to each of the leaf nodes SW may transmit the minimum identification information so that the identification information is covered throughout the entire group, as described in the embodiment of Fig. 6.

[0096] Fig. 8 is an explanatory diagram showing an example of collective communication by a computer system in another embodiment of the present invention. The same elements as those in Figs. 1, 2, 5 and 7 are given the same reference numerals, and detailed explanations are omitted. The hardware configuration and tree structure of the computer system SYS5 shown in Fig. 8 are the same as those of the computer system SYS2 in Fig. 5. Fig. 8 shows an example of a control method for the computer system SYS5 together with the configuration of the computer system SYS5.

[0097] In the computer system SYS5, the identification information sent from each CPU and node AC that is the target of collective communication includes relative height information corresponding to a relative height instead of the height information described in FIG. 7. Therefore, each node SW is not identified by the height information. Relative height information "0" indicates the own node. Relative height information "1" indicates the node one level higher. Relative height information "2" indicates the node two levels higher.

[0098] For example, based on receiving the identification information "(0,(a,c,d))", the node SW of the server SV1 degenerates the data received at ports a, c, d and stores the identification information "(0,(a,c,d))" in a register REG or the like. In addition, the node SW of the server SV1 selects the identification information "(1,(a,b,d))" and "(2,(a,b))" from the identification information received at ports a, c, d, excluding the identification information "(0,(a,c,d))" used to identify the lower node AC.

[0099] Furthermore, the node SW of server SV1 generates new identification information "(0,(a,b,d))" and "(1,(a,b))" by converting the relative height information included in the selected identification information into relative height information with a relative height that is "1" less. Then, the node SW of server SV1 transmits the generated new identification information "(0,(a,b,d))" and "(1,(a,b))" to the upper node SW together with the degenerated data and command code. The node SW of servers SV2 and SV3 operate in the same way as the node SW of server SV1.

[0100] The node SW receiving the identification information "(0,(a,b,d))" or "(1,(a,b))" degenerates the data received at ports a, b, and d based on the received identification information "(0,(a,b,d))" and stores the identification information "(0,(a,b,d))" in a register REG or the like. In addition, the node SW receiving the identification information "(0,(a,b,d))" or "(1,(a,b))" selects the identification information "(1,(a,b))" from the identification information received at ports a, b, and d, excluding the identification information "(0,(a,b,d))" used to identify the lower node AC.

[0101] Furthermore, the node SW receiving the identification information "(0,(a,b,d))" or "(1,(a,b))" generates new identification information "(0,(a,b))" by converting the relative height information included in the selected identification information into relative height information with a relative height that is "1" less. Then, the node SW receiving the identification information "(0,(a,b,d))" or "(1,(a,b))" transmits the generated new identification information "(0,(a,b))" to the upper node SW together with the degenerated data and the instruction code.

[0102] Node SW, which is the root node, reduces the data received at ports a and b based on receiving the identification information "(0,(a,b))" and stores the identification information "(0,(a,b))" in a register REG or the like. Furthermore, since node SW, which is the root node, does not receive any identification information indicating a hierarchy above its own node, it recognizes itself as the root node. Therefore, node SW, which is the root node, transmits the reduced data to lower node SW via ports a and b indicated by the stored identification information "(0,(a,b))".

[0103] Thereafter, the lower node SW that has received the condensed data transmits the condensed data to the participating nodes of the collective communication, among the next lower node SW or node AC, in the same manner as described in FIG. 2 and FIG. 7. For example, node SW connected to servers SV1-SV3 transmits the condensed data transmitted from node SW, which is the root node, to the lower node SW via ports a, b, and d indicated by the stored identification information "(0,(a,b,d))". For example, node SW of server SV1 transmits the condensed data transmitted from the higher node SW to the lower node AC via ports a, c, and d indicated by the stored identification information "(0,(a,c,d))". Then, the collective communication is completed.

[0104] As described above, this embodiment can also provide the same effects as the above-mentioned embodiment. For example, the computer system SYS5 can realize flexible collective communication using an arbitrary tree structure without setting in advance in each switch SW the allocation information of the nodes that perform collective communication. As a result, the computer system SYS5 can realize collective communication with low latency compared to the case where the computer system SYS5 searches a table or the like to recognize the participating nodes of collective communication.

[0105] Furthermore, in this embodiment, collective communication can be performed by mutually transmitting the relative height information of each node SW participating in collective communication and the port information of the port connected to the lower node SW (or AC) in each node SW as identification information between the participating nodes in the collective communication. Each node SW may convert the relative height information included in the identification information used to identify the lower node into relative height information with a relative height less than "1", thereby generating new identification information with relative height information less than 0, and may also transmit this identification information to the higher node SW.

[0106] In Fig. 8, the group of participating nodes AC connected to each of the leaf nodes SW transmits the same identification information in parallel. However, the group of participating nodes AC connected to each of the leaf nodes SW may transmit the minimum identification information so that the identification information is covered throughout the entire group, as described in the embodiment of Fig. 6.

[0107] A part or all of each device (accelerator AC or network switch SW) in the above-mentioned embodiment may be configured with hardware, or may be configured with information processing of software (program) executed by a CPU or a GPU (Graphics Processing Unit), etc. In the case of being configured with information processing of software, software for realizing at least a part of the functions of each device in the above-mentioned embodiment may be stored in a non-transient storage medium (non-transient computer-readable medium) such as a CD-ROM (Compact Disc-Read Only Memory) or a USB (Universal Serial Bus) memory, and the information processing of the software may be executed by reading the software into a computer. The software may also be downloaded via a communication network. Furthermore, the information processing by the software may be executed by hardware by implementing all or a part of the processing of the software in a circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0108] The storage medium that stores the software may be a removable medium such as an optical disk, or a fixed storage medium such as a hard disk or memory. The storage medium may be provided inside the computer (main storage device, auxiliary storage device, etc.) or outside the computer.

[0109] 9 is a block diagram showing an example of a hardware configuration of the accelerator AC or the network switch SW in the above-mentioned embodiment. Each device may be realized as a computer 7 including, for example, a processor 71, a main storage device 72 (memory), an auxiliary storage device 73 (memory), a network interface 74, and a device interface 75, which are connected via a bus 76.

[0110] Although the computer 7 in FIG. 9 includes one of each component, it may include a plurality of the same components. Although one computer 7 is shown in FIG. 9, the software may be installed in a plurality of computers, and each of the plurality of computers may execute the same or different parts of the software. In this case, it may be a form of distributed computing in which each computer communicates via a network interface 74 or the like to execute the processing. That is, each device (accelerator AC or network switch SW) in the above-mentioned embodiment may be configured as a system that realizes a function by one or more computers executing instructions stored in one or more storage devices. Also, it may be configured such that information transmitted from a terminal is processed by one or more computers provided on the cloud, and the processing results are transmitted to the terminal.

[0111] Various calculations of each device (accelerator AC or network switch SW) in the above-mentioned embodiment may be executed in parallel using one or more processors, or using multiple computers via a network. Also, some or all of the processes, means, etc. of the present disclosure may be realized by at least one of a processor and a storage device provided on a cloud that can communicate with the computer 7 via a network. In this way, each device in the above-mentioned embodiment may be in the form of parallel computing using one or more computers.

[0112] The processor 71 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, etc.) that at least controls a computer or performs calculations. The processor 71 may be any of a general-purpose processor, a dedicated processing circuit designed to perform a specific calculation, or a semiconductor device including both a general-purpose processor and a dedicated processing circuit. The processor 71 may also include an optical circuit, or may include a calculation function based on quantum computing.

[0113] The processor 71 may perform arithmetic processing based on data or software input from each device or the like configured inside the computer 7, and may output arithmetic results or control signals to each device or the like. The processor 71 may control each component constituting the computer 7 by executing the OS (Operating System) of the computer 7, applications, or the like.

[0114] Each device (accelerator AC or network switch SW) in the above-described embodiment may be realized by one or more processors 71. Here, the processor 71 may refer to one or more electronic circuits arranged on one chip, or may refer to one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, the electronic circuits may communicate with each other by wire or wirelessly.

[0115] The main memory device 72 may store instructions executed by the processor 71 and various data, and information stored in the main memory device 72 may be read by the processor 71. The auxiliary memory device 73 is a memory device other than the main memory device 72. Note that these memory devices refer to any electronic components capable of storing electronic information, and may be semiconductor memories. The semiconductor memories may be either volatile or non-volatile memories. The memory devices for saving various data in each device (accelerator AC or network switch SW) in the above-mentioned embodiments may be realized by the main memory device 72 or the auxiliary memory device 73, or may be realized by an internal memory built into the processor 71. For example, the memory units such as the register REG in the above-mentioned embodiments may be realized by the main memory device 72 or the auxiliary memory device 73.

[0116] When each device (accelerator AC or network switch SW) in the above-mentioned embodiment is composed of at least one storage device (memory) and at least one processor connected (coupled) to the at least one storage device, at least one processor may be connected to one storage device. At least one storage device may be connected to one processor. A configuration in which at least one processor among a plurality of processors is connected to at least one storage device among a plurality of storage devices may also be included. This configuration may also be realized by storage devices and processors included in a plurality of computers. Furthermore, a configuration in which a storage device is integrated with a processor (for example, a cache memory including an L1 cache and an L2 cache) may also be included.

[0117] The network interface 74 is an interface for connecting to the communication network 8 wirelessly or by wire. The network interface 74 may be an appropriate interface, such as one conforming to an existing communication standard. The network interface 74 may exchange information with an external device 9A connected via the communication network 8. The communication network 8 may be any one of a wide area network (WAN), a local area network (LAN), a personal area network (PAN), etc., or a combination thereof, as long as information is exchanged between the computer 7 and the external device 9A. An example of a WAN is the Internet, an example of a LAN is IEEE802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or Near Field Communication (NFC).

[0118] The device interface 75 is an interface such as a USB that directly connects to the external device 9B.

[0119] The external device 9A is a device connected to the computer 7 via a network. The external device 9B is a device connected directly to the computer 7.

[0120] The external device 9A or the external device 9B may be, for example, an input device. The input device is, for example, a device such as a camera, a microphone, a motion capture device, various sensors, a keyboard, a mouse, a touch panel, or the like, and provides acquired information to the computer 7. Alternatively, the input device may be a device equipped with an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0121] Moreover, the external device 9A or the external device 9B may be, for example, an output device. The output device may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) panel, or a speaker that outputs sound or the like. Alternatively, the output device may be a device including an output unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0122] Furthermore, the external device 9A or the external device 9B may be a storage device (memory). For example, the external device 9A may be a network storage or the like, and the external device 9B may be a storage device such as an HDD.

[0123] Furthermore, the external device 9A or the external device 9B may be a device having some of the functions of the components of each device (accelerator AC or network switch SW) in the above-mentioned embodiment. In other words, the computer 7 may transmit some or all of the processing results to the external device 9A or the external device 9B, or may receive some or all of the processing results from the external device 9A or the external device 9B.

[0124] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. It may also include multiple instances of any element, such as aa, abb, aabbcc, etc. Furthermore, it also includes the addition of elements other than the enumerated elements (a, b, and c), such as having d, as in abcd.

[0125] In this specification (including claims), when expressions such as "using / using data as input / based on / according to / in response to data" (including similar expressions) are used, unless otherwise specified, this includes the use of data itself or data that has been processed in some way (e.g., data with noise added, normalized data, feature values ​​extracted from data, intermediate representations of data, etc.). In addition, when it is stated that a result is obtained "using data as input / based on / according to / in response to data" (including similar expressions), unless otherwise specified, this includes the use of the result based only on the data, or the use of the result influenced by other data, factors, conditions, and / or states other than the data. In addition, when it is stated that "data is output" (including similar expressions), unless otherwise specified, this includes the use of data itself as output, or data that has been processed in some way (e.g., data with noise added, normalized data, feature values ​​extracted from data, intermediate representations of various data, etc.) as output.

[0126] In this specification (including the claims), the terms "connected" and "coupled" are intended as open-ended terms including any of direct connection / coupling, indirect connection / coupling, electrically connection / coupling, communicatively connection / coupling, functionally connection / coupling, physically connection / coupling, etc. The terms should be interpreted appropriately depending on the context in which the terms are used, but any connection / coupling form that is not intentionally or naturally excluded should be interpreted as being included in the terms without any restriction.

[0127] In this specification (including the claims), when the expression "A configured to B" is used, it may include that the physical structure of element A has a configuration capable of performing operation B, and that the permanent or temporary setting / configuration of element A is configured / set to actually perform operation B. For example, when element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and is configured to actually perform operation B by setting a permanent or temporary program (instruction). Also, when element A is a dedicated processor or dedicated arithmetic circuit, it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0128] In this specification (including the claims), when terms implying containing or possessing (such as "comprising / including" and "having") are used, they are intended to be open-ended terms that include the case of containing or possessing things other than the object indicated by the object of the term. When the object of such terms implying containing or possessing does not specify a quantity or suggests a singular number (such as an expression using the article "a" or "an"), the expression should be construed as not being limited to a specific number.

[0129] In this specification (including the claims), even if expressions such as "one or more" and "at least one" are used in some places and expressions that do not specify a quantity or suggest a singular number (expressions using the articles "a" or "an") are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or suggest a singular number (expressions using the articles "a" or "an") should be interpreted as not necessarily being limited to a specific number.

[0130] In this specification, when a particular advantage / result is described as being obtained from a particular configuration of an embodiment, it should be understood that the same effect can also be obtained from one or more other embodiments having the same configuration, unless there is a reason to the contrary. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or states, and that the effect is not necessarily obtained by the configuration. The effect is merely obtained by the configuration described in the embodiment when various factors, conditions, and / or states are satisfied, and the effect is not necessarily obtained in the invention according to the claim that specifies the configuration or a similar configuration.

[0131] In this specification (including claims), when a plurality of pieces of hardware perform a predetermined process, each piece of hardware may cooperate to perform the predetermined process, or a portion of the hardware may perform all of the predetermined process. Also, a portion of the hardware may perform a portion of the predetermined process, and another piece of hardware may perform the remainder of the predetermined process. In this specification (including claims), when an expression such as "one or more pieces of hardware perform a first process, and the one or more pieces of hardware perform a second process" (including similar expressions) is used, the hardware performing the first process and the hardware performing the second process may be the same or different. In other words, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more pieces of hardware. The hardware may include an electronic circuit or a device including an electronic circuit.

[0132] In this specification (including the claims), when multiple storage devices (memories) store data, each of the multiple storage devices may store only a portion of the data, or may store the entire data. Also, a configuration in which only some of the multiple storage devices store data may be included.

[0133] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the above-mentioned individual embodiments. Various additions, modifications, replacements, partial deletions, etc. are possible within the scope of the conceptual idea and intent of the present invention derived from the contents defined in the claims and their equivalents. For example, in the above-mentioned embodiments, when numerical values ​​or formulas are used for explanation, these are shown for illustrative purposes and do not limit the scope of the present disclosure. In addition, the order of each operation shown in the embodiment is also illustrative and does not limit the scope of the present disclosure.

Claims

1. A computer system having a plurality of first nodes connected in a tree shape, each having a computing function, and at least one second node connected to a leaf side of the first nodes, A second participating node, which is the second node participating in the collective communication, transmits target data of the collective communication and identification information for identifying the node participating in the collective communication to the first node; Each of the first participating nodes, which is the first nodes participating in the collective communication, performs a calculation on the target data received from the second participating node, and transmits, to a higher-level node, identification information for identifying the nodes participating in the collective communication that is generated based on at least a part of the identification information received from the second participating node or at least a part of the identification information received from the second participating node, together with the calculation result. Computer system.

2. The identification information corresponds to a node on a one-to-one basis and is different for each node.

2. The computer system of claim 1.

3. the second participating node generates a packet in which the target data and the identification information are stored in a payload and a packet header, respectively, and transmits the packet to the first node; The first participating node acquires the target data and the identification information from the payload and the packet header of the packet received from the second participating node, respectively.

3. The computer system according to claim 1 or 2.

4. the first node is a network switch; The second node is a processor or an accelerator. A computer system according to any one of claims 1 to 3.

5. The group of the second participating nodes connected to the leaf side of the first participating node transmits identification information for identifying the second participating nodes belonging to the group and identification information for identifying the first participating node.

5. The computer system according to claim 1.

6. A group of second participating nodes connected to the leaf side of the first participating node transmits height information corresponding to the height on the tree and port information identifying a port for receiving data as identification information of each of the first participating nodes sequentially connected in correspondence with the group; Each of the first participating nodes identifies a lower node based on the height information and port information of the node.

6. The computer system of claim 5.

7. the height information transmitted by the second participating node indicates a relative height from the second participating node; Each of the first participating nodes converts the height information received from the lower node into height information corresponding to a relative height from the node itself, and transmits the converted height information together with port information to the upper node.

7. The computer system of claim 6.

8. Second participating nodes in the group transmit the same identification information to each other.

8. The computer system according to claim 5, wherein the first and second processors are connected to the first and second processors.

9. A second participating node in the group transmits identification information such that the identification information of the second participating node to be transmitted is covered within its own group, and the identification information of the first participating node to be transmitted is covered within all groups.

8. The computer system according to claim 5, wherein the first and second processors are connected to the first and second processors.

10. The second participating node transmits identification information excluding identification information for identifying the first participating node located at the root to a participating node one level above; A first participating node that does not receive identification information of a node at a level higher than the node itself transmits a calculation result to a lower first participating node; The lower-level first participating node that has received the operation result transmits the operation result to the lower-level first participating node; The first participating node on the leaf side that has received the calculation result transmits the received calculation result to the second participating node.

10. The computer system according to claim 1.

11. Each of the first participating nodes stores, among the received identification information, identification information that identifies a node lower than the node itself, and transmits a calculation result to a node indicated by the stored identification information.

11. The computer system of claim 10.

Images