Accelerator cluster super node device and computing acceleration device

Abstract

The utility model relates to an accelerator cluster super-node device and a computing acceleration device. The accelerator cluster super-node device comprises a southbound PCIe switch and at least one artificial intelligence server. Each artificial intelligence server comprises at least two acceleration card groups, and each acceleration card group comprises at least one acceleration card; the southbound PCIe switch is coupled to the acceleration cards in the at least two acceleration card groups in the at least one artificial intelligence server to form a southbound PCIe connection path across the acceleration card groups between the acceleration cards. According to the invention, southbound connection between acceleration cards in different acceleration card groups and different server nodes is realized, the waiting time is shorter than that of northbound data communication, the data receiving waiting time between different acceleration cards in different acceleration card groups during data interaction is reduced, direct interconnection communication of the acceleration cards is realized, and the communication efficiency is improved. The data receiving waiting time of the acceleration card can be shortened, and the calculation speed can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of server communication technology, and in particular to an accelerator cluster supernode device and a computing acceleration device. Background Art

[0002] Computing servers, especially computing servers used for AI training and reasoning (also called AI servers), require a large amount of computing. They usually use a structure that integrates multiple accelerator cards and a central processor for resource allocation and data scheduling. Each accelerator card in the same accelerator card group is responsible for its own computing tasks, and sends and receives data through the internal switch to achieve data interaction with other accelerator cards. In addition, in the computing server, the internal switch also establishes a connection with the outside world through the NIC (Network Interface Card). Different computing servers are interconnected through their respective NICs and network switches to achieve data interaction and data upload between computing servers via the network switch.

[0003] In this way, if different accelerator cards in different accelerator card groups want to exchange data, they must at least go through the communication path of the relevant in-machine switch and the central processor to achieve this. This will increase the waiting time for the accelerator card to receive data due to the delay in the communication information transmission between the in-machine switch and the central processor chip, thereby limiting the computing speed of the computing server as a whole. Different accelerator cards in different computing servers also need to go through the relevant NIC and network switch to achieve data exchange. Because of the delay in the communication information transmission between the NIC and the network switch, and the possible distance between the network switch and the NIC, the accelerator card has a longer waiting time to receive data, which further limits the overall computing speed of the supercomputing node composed of the network switch and multiple computing servers. Utility Model Content

[0004] In view of this, the present disclosure provides an accelerator cluster supernode device and a computing acceleration device to help reduce the data reception waiting time during data interaction between different acceleration cards in different acceleration card groups, thereby helping to improve the computing speed of computing servers and supercomputing nodes.

[0005] The technical solution of the present disclosure is achieved as follows:

[0006] According to one aspect of an embodiment of the present disclosure, an accelerator cluster supernode device is provided, including:

[0007] A southbound PCIe switch and at least one artificial intelligence server; wherein,

[0008] Each of the artificial intelligence servers includes at least two acceleration card groups, and each of the acceleration card groups contains at least one acceleration card;

[0009] The southbound PCIe switch is coupled to the accelerator cards in at least two accelerator card groups in at least one of the artificial intelligence servers to form a southbound PCIe connection path between the accelerator cards across the accelerator card groups.

[0010] In a possible implementation manner, when each of the acceleration card groups contains at least two acceleration cards, the southbound PCIe switch is coupled to at least one acceleration card in each of the acceleration card groups.

[0011] In one possible implementation, each of the artificial intelligence servers further includes:

[0012] at least two processors, wherein in each of the artificial intelligence servers, the number of the processors is the same as the number of the acceleration card groups; and,

[0013] At least two northbound PCIe switches, in each of the artificial intelligence servers, the number of the northbound PCIe switches is the same as the number of the acceleration card groups, each of the northbound PCIe switches is coupled one-to-one to each of the acceleration card groups, each of the northbound PCIe switches is coupled to at least one acceleration card in the coupled acceleration card group, and each of the northbound PCIe switches is coupled one-to-one to each of the processors to form a northbound PCIe connection path for the acceleration card group.

[0014] In one possible implementation, in each of the artificial intelligence servers, the two acceleration card groups are northbound interconnected via the processors to which they are respectively coupled.

[0015] In one possible implementation, each of the artificial intelligence servers further includes:

[0016] NIC card, in each of the artificial intelligence servers, the NIC card is interconnected with the at least two northbound PCIe switches.

[0017] In a possible implementation manner, the accelerator cluster supernode device further includes:

[0018] A network switch, wherein the network switch is interconnected with the NIC card in each of the artificial intelligence servers. When there are at least two artificial intelligence servers, the network switch forms a northbound PCIe connection path for the acceleration card group via the NIC card and the network switch between at least two of the artificial intelligence servers.

[0019] In a possible implementation manner, the southbound PCIe switch includes:

[0020] At least one southbound PCIe sub-switch, the total number of ports of all the southbound PCIe sub-switches being not less than the number of ports of the acceleration card coupled to the southbound PCIe switch in the accelerator cluster supernode device.

[0021] In one possible implementation, the acceleration card groups are paired in pairs to form a computing acceleration unit in an artificial intelligence server.

[0022] In a possible implementation, between two acceleration card groups in one of the artificial intelligence servers, any two designated acceleration cards in different acceleration card groups are directly connected to the south.

[0023] In a possible implementation, the southbound PCIe switch and the acceleration card are connected via at least one of an optical module and a copper cable.

[0024] According to another aspect of an embodiment of the present disclosure, a computing acceleration device is provided, including:

[0025] at least two acceleration card groups, each of which contains at least one acceleration card; and

[0026] A southbound PCIe switch is coupled to the acceleration cards in the at least two acceleration card groups to form a southbound PCIe connection path between the acceleration cards across the acceleration card groups.

[0027] It can be seen from the above scheme that the accelerator cluster supernode device and computing acceleration device of the present invention realize the southbound PCIe connection between the accelerator cards in different accelerator card groups, thereby realizing data communication across accelerator card groups. Compared with the northbound data communication based on the northbound connection structure, the data reception waiting time of the accelerator card is shorter, which helps to reduce the data reception waiting time between different accelerator cards in different accelerator card groups during data interaction. On this basis, when the computing servers and supercomputing nodes using the accelerator cluster supernode device and computing acceleration device structure of the present invention perform computing tasks, the data interaction of the accelerator cards between the accelerator card groups using the southbound PCIe connection path across the accelerator card group constructed by the southbound PCIe switch will help shorten the waiting time for the accelerator card to receive data, help improve the computing speed of computing servers (such as servers for artificial intelligence reasoning computing) and supercomputing nodes, and can realize TP8, TP16, TP32 and other parallel computing of multiple accelerator cards. The accelerator cluster supernode device and computing acceleration device disclosed in the present invention realize point-to-point data transmission across computing servers, across central processing units, and across PCIe Domains based on the southbound PCIe switch, realize southbound interconnection of multiple accelerator cards in a multi-accelerator card group, and provide hardware support for multi-card parallel computing. Based on the accelerator cluster supernode device and computing acceleration device disclosed in the present invention, multiple computing servers can realize interconnection of unlimited number of accelerator cards through the southbound PCIe switch. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1A It is a structural diagram of a computing server in the related art;

[0029] Figure 1B It is a schematic diagram of another type of connection structure inside a computing server in the related art;

[0030] Figure 1C It is a structural diagram of another connection inside a computing server in the related art;

[0031] Figure 2 It is a schematic diagram of the structure of a supercomputing node composed of two computing servers;

[0032] Figure 3 is a schematic diagram of the structure of a computing acceleration device according to an exemplary embodiment;

[0033] Figure 4 is a schematic diagram of a northbound connection-related structure of an acceleration card group according to an exemplary embodiment;

[0034] Figure 5 is a schematic structural diagram of another computing acceleration device according to an exemplary embodiment;

[0035] Figure 6 is a schematic structural diagram of another computing acceleration device according to an exemplary embodiment;

[0036] Figure 7 is a schematic structural diagram of an accelerator cluster supernode device according to an exemplary embodiment;

[0037] Figure 8 It is a structural diagram of an application scenario embodiment 1 of the embodiment of the present disclosure;

[0038] Fig. 9 It is a structural diagram of application scenario embodiment 2 of the embodiment of the present disclosure;

[0039] Fig.10 is a structural diagram of application scenario embodiment 3 of the embodiment of the present disclosure;

[0040] Fig.11 It is a simplified structural diagram of application scenario embodiment 4 of the embodiment of the present disclosure;

[0041] Fig.12 is a structural diagram of application scenario embodiment 5 of the embodiments of the present disclosure;

[0042] Fig.13 It is a structural diagram of application scenario embodiment 6 of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below with reference to the accompanying drawings and examples.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0045] The direct interconnection of AI acceleration cards in related technologies is limited to direct interconnection within a single machine and a single node, and cannot achieve direct interconnection across nodes. The main solution for multi-machine interconnection is usually to achieve multi-machine interconnection through RDMA (Remote Direct Memory Access) of NIC (Network Interface Card) / IB (InfiniBand) network cards, which has the disadvantages of large latency and low communication efficiency. Multi-machine communication through network interconnection not only has technical limitations, but also requires more network cards and network switches at the system level, resulting in high equipment cost investment.

[0046] Figure 1A1 is a schematic diagram of a computing server in the related art. The computing server 100 is a server for artificial intelligence reasoning computing, such as Figure 1A As shown, the computing server 100 mainly includes two artificial intelligence acceleration card groups 1, two in-machine PCIe (Peripheral Component Interconnect Express) switches and two CPUs (Central Processing Units), wherein each artificial intelligence acceleration card group 1 includes four acceleration cards, each of which contains a GPU (Graphics Processing Unit), and the four GPUs are used as computing chips. Figure 1A As shown, the GPUs in the four accelerator cards in the left artificial intelligence acceleration card group 1 are named GPU0, GPU1, GPU2 and GPU3 respectively, and the four GPUs in the four accelerator cards in the right artificial intelligence acceleration card group 1 are named GPU4, GPU5, GPU6 and GPU7 respectively. The four accelerator cards in the same artificial intelligence acceleration card group 1 are directly connected point-to-point (P2P) with each other. At the same time, the four accelerator cards in the same artificial intelligence acceleration card group 1 are also coupled to the corresponding in-machine PCIe switch, and the in-machine PCIe switch is coupled to the corresponding CPU. The communication between the four accelerator cards and the CPU in the same artificial intelligence acceleration card group 1 is realized through the corresponding in-machine PCIe switch, and the two artificial intelligence acceleration card groups 1 communicate through their respective corresponding CPUs. In addition, in order to realize data communication with the outside world, the computing server 100 also includes a NIC, and the two in-machine PCIe switches are coupled to the NIC, so that the accelerator cards and CPUs in the artificial intelligence acceleration card group 1 can communicate data with the outside world through the in-machine PCIe switch and the NIC. It should be noted that for the sake of schematic simplicity, two NICs are presented in the figure, and in reality, these two NICs can be one NIC.

[0047] It can be seen that the data interaction between any two accelerator cards in the two AI accelerator card groups 1 needs to be implemented through their corresponding in-machine PCIe switches and CPUs. For example Figure 1A As shown in the figure, the accelerator card of CPU0 and the accelerator card of CPU7 need to be connected through the PCIe switch and CPU on the left side and the PCIe switch and CPU on the right side. Because the PCIe switch and CPU on the inside have bandwidth and latency limitations, compared with the accelerator card with point-to-point direct interconnection, the waiting time for the accelerator card to receive data will be increased. Therefore, when the server executes computing tasks, the accelerator card data interaction between the artificial intelligence acceleration card group 1 will restrict the computing efficiency of the server.

[0048] remove Figure 1A In addition to the connection structure shown, there may be other forms of connection between accelerator cards and between accelerator cards and the PCIe switch in the machine, such as Figure 1B , Figure 1C The connection structure shown.

[0049] Figure 1B In the structure shown, four in-machine PCIe switches are included. GPU0 and GPU1 in the left artificial intelligence acceleration card group 1 are connected to the left CPU through one of the in-machine PCIe switches, GPU2 and GPU3 in the left artificial intelligence acceleration card group 1 are connected to the left CPU through one of the in-machine PCIe switches, GPU4 and GPU5 in the right artificial intelligence acceleration card group 1 are connected to the right CPU through one of the in-machine PCIe switches, and GPU6 and GPU7 in the right artificial intelligence acceleration card group 1 are connected to the right CPU through one of the in-machine PCIe switches. In addition to the direct interconnection between the four accelerator cards in the artificial intelligence acceleration card group 1, GPU0 in the left artificial intelligence acceleration card group 1 is directly interconnected with GPU6 in the right artificial intelligence acceleration card group 1, GPU1 in the left artificial intelligence acceleration card group 1 is directly interconnected with GPU7 in the right artificial intelligence acceleration card group 1, GPU2 in the left artificial intelligence acceleration card group 1 is directly interconnected with GPU4 in the right artificial intelligence acceleration card group 1, and GPU3 in the left artificial intelligence acceleration card group 1 is directly interconnected with GPU5 in the right artificial intelligence acceleration card group 1.

[0050] Figure 1C In the structure shown, any two acceleration cards in the left artificial intelligence acceleration card group 1 and the right artificial intelligence acceleration card group 1 are directly interconnected.

[0051] remove Figure 1A , Figure 1B , Figure 1C In addition to the structure shown, there may be other forms of interconnection topologies between acceleration cards and between an acceleration card and an internal PCIe switch, which will not be described in detail.

[0052] When deploying a GPU server cluster, several computing servers are usually combined into a supercomputing node to meet the needs of larger computing tasks. Figure 2 This is a schematic diagram of the structure of a supercomputing node composed of two computing servers. Figure 2As shown, the supercomputing node 400 includes two computing servers 100 and a network switch 200, and the NIC in the computing server 100 is coupled to the network switch 200. In the supercomputing node 400, the data interaction path between any two accelerator cards respectively located in the two computing servers 100 includes at least the in-machine PCIe switch in each computing server 100, the NIC in each computing server 100, and the network switch 200. Therefore, the data interaction between any two accelerator cards respectively located in the two computing servers 100 is at least limited by the bandwidth and latency of the network switch 200 and the two NICs on the data interaction path. Therefore, when the supercomputing node 400 performs computing tasks, the data interaction between the accelerator cards in different computing servers 100 in the supercomputing node 400 will also restrict the computing efficiency of the server.

[0053] In view of this, the embodiments of the present disclosure provide an accelerator cluster supernode device and a computing acceleration device to help reduce the data reception waiting time between different acceleration cards in different acceleration card groups during data interaction, thereby helping to improve the computing speed of computing servers and supercomputing nodes, realize parallel computing of multiple acceleration cards, and improve cluster performance.

[0054] Figure 3 is a schematic diagram showing the structure of a computing acceleration device according to an exemplary embodiment, such as Figure 3 As shown, in the exemplary embodiment, the computing acceleration device 300 mainly includes an acceleration card group 301 and a southbound PCIe switch 302. The number of the acceleration card groups 301 is at least two, and each acceleration card group 301 contains at least one acceleration card, for example Figure 3 As shown, each accelerator card group 301 contains four accelerator cards. The southbound PCIe switch 302 is coupled to the accelerator cards in at least two accelerator card groups 301, forming a southbound PCIe connection path between the accelerator cards across the accelerator card groups 301 (eg, Figure 3 shown by the dashed line).

[0055] In an exemplary embodiment, the accelerator card may be an artificial intelligence accelerator card, which contains a computing accelerator chip. In an exemplary embodiment, the computing accelerator chip may be any one of the processors that can be used for computing, such as a GPU (Graphics Processing Unit), a TPU (Tensor Processing Unit), an NPU (Neural Network Processing Unit), a DPU (Deep Learning Processing Unit), an APU (Accelerated Processing Unit), and a GPGPU (General-Purpose computing on Graphics Processing Unit).

[0056] In the exemplary embodiment, "south" is relative to "north", where "north" refers to Figure 1A , Figure 2In the structure shown, the accelerator card is connected to the PCIe switch in the machine, wherein the connection between the accelerator card and the PCIe switch in the machine is a northbound connection. On this basis, the structure in which the accelerator card is coupled to the CPU, NIC, and network switch 200 through the PCIe switch in the machine all belongs to a northbound connection structure. In the embodiment of the present disclosure, the southbound PCIe switch 302 is not coupled to devices or equipment such as the CPU, NIC, and network switch 200. The southbound PCIe switch 302 is only coupled to the accelerator card, and the southbound PCIe connection path does not include devices or equipment such as the CPU, NIC, and network switch 200. Based on the computing acceleration device 300 of the embodiment of the present disclosure, the acceleration cards in different acceleration card groups 301 can communicate data via the southbound PCIe connection path of the southbound PCIe switch 302. Compared with northbound data communication based on the northbound connection structure, the data between the acceleration cards only passes through the southbound PCIe switch 302. Therefore, the data reception waiting time of the acceleration card is shorter than the data reception waiting time when northbound data communication is adopted, which helps to reduce the data reception waiting time when different acceleration cards in different acceleration card groups 301 are interacting with each other. On this basis, when the computing server and supercomputing node using the computing acceleration device structure of the embodiment of the present disclosure perform computing tasks, the data interaction of the acceleration cards between the acceleration card groups 301 using the southbound PCIe connection path across the acceleration card group 301 constructed by the southbound PCIe switch 302 will help shorten the waiting time for the acceleration card to receive data, help improve the computing speed of the computing server (such as a server for artificial intelligence reasoning computing) and the supercomputing node, and realize TP8, TP16, TP32 and other parallel computing of multiple acceleration cards. Among them, TP (TensorParallelism) is called tensor parallelism, which is a parallel strategy for large-scale model training. It is mainly used to solve the problem that a single device cannot accommodate the entire model parameters. The basic idea of ​​tensor parallelism is to decompose the model parameters or computing tasks onto multiple devices. Each device is only responsible for a part of the parameters or computing tasks, thereby realizing distributed training of the model. For example, TP8 means decomposing the model parameters or computing tasks onto 8 devices, TP16 means decomposing the model parameters or computing tasks onto 16 devices, TP32 means decomposing the model parameters or computing tasks onto 32 devices, and so on.

[0057] In addition, in the disclosed embodiment, the southbound PCIe switch 302 adopts the public PCIe protocol, which has a wider scope of application and greater versatility than private protocols, and helps to expand the use scenarios and application scope of acceleration card interconnection.

[0058] Usually, the same acceleration card group 301 will contain more than one acceleration card at the same time. For example, in the related technology, each artificial intelligence acceleration card group 1 includes four acceleration cards, and the acceleration cards in an acceleration card group 301 will be directly interconnected by point-to-point (P2P). For example, in the related technology, the four acceleration cards in the same artificial intelligence acceleration card group 1 are directly interconnected by point-to-point. However, based on the needs of computing tasks, there may be situations where different acceleration cards in multiple acceleration card groups 301 need to coordinate with each other to complete corresponding computing tasks. Therefore, preferably, the southbound PCIe switch 302 needs to be coupled to each acceleration card. However, it may be limited by the number of ports of the southbound PCIe switch 302 itself, and the southbound PCIe switch 302 may not be able to fully connect to all acceleration cards in all acceleration card groups 301.

[0059] Based on this, in the exemplary embodiment, when each accelerator card group 301 contains at least two accelerator cards, the southbound PCIe switch 302 is coupled to at least one accelerator card in each accelerator card group 301. The accelerator card that needs to be connected to the southbound PCIe switch 302 can be determined and set according to computing tasks and design requirements.

[0060] In this way, when the number of southbound PCIe switch 302 ports is limited, the accelerator cards can be connected to the maximum extent possible in combination with computing tasks and design requirements. This helps to maintain a short waiting time for receiving data between accelerator cards across accelerator card groups that need to interact with data when the number of southbound PCIe switch 302 ports is limited, and helps to maintain the computing speed of computing servers and supercomputing nodes.

[0061] Figure 4 FIG. 1 is a schematic diagram of a northbound connection-related structure of an acceleration card group according to an exemplary embodiment. Figure 4 Shown and combined Figure 3 As shown, in the exemplary embodiment, the computing acceleration device of the embodiment of the present disclosure further includes at least two processors and at least two northbound PCIe switches (i.e. Figure 1A , Figure 2 The number of processors is the same as the number of acceleration card groups 301, and the number of northbound PCIe switches is the same as the number of acceleration card groups 301. Each northbound PCIe switch is coupled to at least one acceleration card in the coupled acceleration card group 301 (for example, each northbound PCIe switch is coupled to each acceleration card in the coupled acceleration card group 301), and each northbound PCIe switch is coupled to each processor one-to-one to form a northbound PCIe connection path of the acceleration card group 301.

[0062] Figure 5is a schematic diagram of the structure of another computing acceleration device according to an exemplary embodiment. Figure 5 As shown, in the exemplary embodiment, the two acceleration card groups 301 in the computing acceleration device 300 are interconnected northward through the processors to which they are coupled. It should be noted that the connection between the acceleration card groups 301 is one-to-one interconnection, that is, the acceleration card groups 301 are interconnected one-to-one through their respective processors. In combination with related technologies, for example Figure 1A In the related art shown, the computing server 100 mainly includes two artificial intelligence acceleration card groups 1, so the two acceleration card groups 301 can be interconnected one-to-one through their respective processors to form a computing server 100 as in the related art. Based on this, in the exemplary embodiment, in the computing acceleration device 300, in at least two acceleration card groups 301, the acceleration card groups 301 are paired in pairs to form a computing acceleration unit in a computing server 100 (artificial intelligence server).

[0063] In the same computing acceleration device 300 (or computing server 100), depending on the computing task, there may be a situation where two designated acceleration cards in different acceleration card groups 301 need to communicate data. In the related art, the two designated acceleration cards will communicate data through the northbound PCIe switch and processor coupled to the acceleration card group 301. In an illustrative embodiment, in the computing acceleration device of the embodiment of the present disclosure, between two acceleration card groups 301 in a computing server 100 (such as an artificial intelligence server), any two designated acceleration cards in different acceleration card groups 301 are directly connected to the south. In an illustrative embodiment, because the two acceleration card groups 301 are in the same computing server 100, the southbound direct connection can be a direct connection of the acceleration card through the corresponding southbound port, without the need to be connected via the southbound PCIe switch 302.

[0064] In this way, any two designated accelerator cards in different accelerator card groups 301 can change from northbound data communication to southbound data communication, thereby reducing the data reception waiting time of the designated accelerator cards during data exchange.

[0065] Figure 6 is a schematic diagram of the structure of another computing acceleration device according to an exemplary embodiment. Figure 6 As shown, in the exemplary embodiment, the computing acceleration device may further include at least two NIC cards. The at least two NIC cards are interconnected with at least two northbound PCIe switches in a one-to-one manner.

[0066] On this basis, if Figure 6As shown, in the exemplary embodiment, the computing acceleration device may further include a network switch 200. The network switch 200 is interconnected with at least two NIC cards, and a northbound PCIe connection path is formed between at least two acceleration card groups 301 via at least two NIC cards and the network switch 200.

[0067] from Figure 5 and Figure 6 It can be seen from the computing acceleration device shown in the illustrated embodiment that the computing acceleration device of the disclosed embodiment can be based on the relevant computing server 100, without changing the overall architecture of the computing server 100, but only needs to expand the port of the acceleration card corresponding to the southbound connection and connect the southbound PCIe switch 302. On this basis, the acceleration card adopts the public PCIe protocol to realize data transmission via the southbound PCIe switch 302.

[0068] In the exemplary embodiment, the number of accelerator cards that can be connected to the southbound PCIe switch 302 may theoretically have no upper limit, but in practice, the number of ports of the PCIe switch may be limited, such as 8 ports, 16 ports, 32 ports, etc. Based on this, in order to achieve the connection of the southbound PCIe switch 302 to any number of accelerator cards, in the exemplary embodiment, the southbound PCIe switch 302 may include at least one southbound PCIe sub-switch, and the total number of ports of all southbound PCIe sub-switches is not less than the number of ports of the accelerator cards in at least two accelerator card groups 301 coupled to the southbound PCIe switch 302. The total number of ports of all southbound PCIe sub-switches is not less than the number of ports of the accelerator cards coupled to the southbound PCIe switch 302.

[0069] In this way, the number of southbound PCIe sub-switches can be expanded to achieve the connection of southbound PCIe links of any number of accelerator cards. The southbound PCIe sub-switches can be implemented by using PCIe switches.

[0070] In the exemplary embodiment, a variety of feasible connection forms can be adopted between the southbound PCIe switch 302 and the acceleration card. For example, the southbound PCIe switch 302 and the acceleration card can be connected through at least one of an optical module and a copper cable.

[0071] The computing acceleration device of the disclosed embodiment realizes point-to-point data transmission of the acceleration card across computing servers, processors, and PCIe Domains based on the southbound PCIe switch, realizes the southbound interconnection of multiple acceleration cards in a multi-acceleration card group, and provides hardware support for parallel computing of multiple acceleration cards. Based on the computing acceleration device of the disclosed embodiment, multiple computing servers can realize interconnection of an unlimited number of acceleration cards through the southbound PCIe switch. The computing acceleration device of the disclosed embodiment can support the plug-and-play function of the optical module or cable between the artificial intelligence acceleration card end and the southbound PCIe switch port. In addition, in the computing acceleration device of the disclosed embodiment, the acceleration cards connected to the same southbound PCIe switch port can use the send-and-play instant communication technology, without the need for additional configuration, which can reduce the workload of configuration.

[0072] In an exemplary embodiment, the present disclosure also provides an accelerator cluster super node device, Figure 7 is a schematic diagram of the structure of the accelerator cluster super node device according to the exemplary embodiment, such as Figure 7 As shown, the accelerator cluster supernode device mainly includes a southbound PCIe switch 302 and at least one artificial intelligence server 700. Each artificial intelligence server 700 includes at least two acceleration card groups 301, and each acceleration card group contains at least one acceleration card. The southbound PCIe switch 302 is coupled to the acceleration cards in at least two acceleration card groups 301 in at least one artificial intelligence server 700, forming a southbound PCIe connection path across the acceleration card group 301 between the acceleration cards, wherein the acceleration card group 301 spanned by the acceleration card can be an acceleration card group 301 in the same artificial intelligence server 700, or can be an acceleration card group 301 in different artificial intelligence servers 700.

[0073] The accelerator cluster supernode device of the embodiment of the present disclosure realizes southbound PCIe connection between accelerator cards in different accelerator card groups in at least one artificial intelligence server, thereby realizing data communication of accelerator cards across accelerator card groups and even across artificial intelligence servers. Compared with northbound data communication based on a northbound connection structure, the data reception waiting time of the accelerator card is shorter, thereby helping to reduce the data reception waiting time during data interaction between different accelerator cards in different accelerator card groups and even in different artificial intelligence servers. On this basis, when a supercomputing node using the accelerator cluster supernode device structure of the embodiment of the present disclosure performs computing tasks, data interaction of accelerator cards between accelerator card groups and even between artificial intelligence servers using the southbound PCIe connection path across accelerator card groups constructed by a southbound PCIe switch will help shorten the waiting time for accelerator cards to receive data, help improve the computing speed of the supercomputing node, and realize TP8, TP16, TP32 and other parallel computing of multiple accelerator cards in the accelerator cluster supernode. The accelerator cluster supernode device of the embodiment of the present disclosure realizes point-to-point data transmission across artificial intelligence servers, central processing units, and PCIeDomains based on the southbound PCIe switch, which helps to realize the southbound interconnection of any accelerator card in the accelerator cluster supernode and provides hardware support for multi-card parallel computing. The accelerator cluster supernode device based on the embodiment of the present disclosure can realize the interconnection of an unlimited number of accelerator cards in the accelerator cluster supernode through the southbound PCIe switch.

[0074] In the exemplary embodiment, in the accelerator cluster supernode device, when each accelerator card group 301 contains at least two accelerator cards, the southbound PCIe switch 302 is coupled to at least one accelerator card in each accelerator card group 301. The accelerator card that needs to be connected to the southbound PCIe switch 302 can be determined and set according to computing tasks and design requirements.

[0075] Continue to see Figure 7 As shown, in the exemplary embodiment, in the accelerator cluster supernode device, each artificial intelligence server 700 also includes at least two processors and at least two northbound PCIe switches. Among them, in each artificial intelligence server 700, the number of processors is the same as the number of acceleration card groups 301. In each artificial intelligence server 700, the number of northbound PCIe switches is the same as the number of acceleration card groups 301, and each northbound PCIe switch is coupled to each acceleration card group 301 one-to-one. In the exemplary embodiment, each northbound PCIe switch is coupled to at least one acceleration card in the coupled acceleration card group 301, and each northbound PCIe switch is coupled to each processor one-to-one to form a northbound PCIe connection path of the acceleration card group 301.

[0076] like Figure 7 As shown, in the exemplary embodiment, in each artificial intelligence server 700, two acceleration card groups 301 are interconnected northbound through their respective coupled processors.

[0077] In the exemplary embodiment, inside each artificial intelligence server 700, especially when the artificial intelligence server 700 includes two acceleration card groups 301, the connection between the acceleration card groups 301 can be one-to-one interconnection, that is, the acceleration card groups 301 are interconnected one-to-one through their respective processors.

[0078] like Figure 7 As shown, in the exemplary embodiment, in the accelerator cluster supernode device, each artificial intelligence server 700 also includes a NIC card, and in each artificial intelligence server 700, the NIC card is interconnected with at least two northbound PCIe switches.

[0079] On this basis, if Figure 7 As shown, in the exemplary embodiment, the accelerator cluster supernode device may further include a network switch 200. The network switch 200 is interconnected with the NIC card in each artificial intelligence server 700. When there are at least two artificial intelligence servers 700, the network switch 200 forms a northbound PCIe connection path of the acceleration card group 301 via the NIC card and the network switch 200 between the at least two artificial intelligence servers 700.

[0080] from Figure 7 It can be seen from the accelerator cluster supernode device shown in the illustrated embodiment that the accelerator cluster supernode device of the embodiment of the present disclosure can be based on the relevant accelerator cluster supernode without changing its overall architecture, but only needs to expand the port corresponding to the southbound connection of the accelerator card and connect the southbound PCIe switch 302. On this basis, the accelerator card adopts the public PCIe protocol to realize data transmission via the southbound PCIe switch 302. Therefore, it helps to reduce the transformation cost of the accelerator cluster supernode.

[0081] In the exemplary embodiment, the number of accelerator cards that can be connected to the southbound PCIe switch 302 may theoretically have no upper limit, but in practice, the number of ports of the PCIe switch may be limited, such as 8 ports, 16 ports, 32 ports, etc. Based on this, in order to achieve the connection of the southbound PCIe switch 302 to any number of accelerator cards, in the exemplary embodiment, the southbound PCIe switch 302 may include at least one southbound PCIe sub-switch, and the total number of ports of all southbound PCIe sub-switches is not less than the number of ports of the accelerator cards coupled to the southbound PCIe switch 302 in the accelerator cluster supernode device.

[0082] In a specific application scenario, in the accelerator cluster supernode device, the acceleration card groups 301 are paired in pairs to form a computing acceleration unit in an artificial intelligence server 700.

[0083] In the same artificial intelligence server 700, depending on the computing task, there may be a situation where two designated accelerator cards in different accelerator card groups 301 need to communicate data. In the related art, the two designated accelerator cards will communicate data through the northbound PCIe switch and processor coupled to the accelerator card group 301. In an illustrative embodiment, in the accelerator cluster supernode device, between two accelerator card groups 301 in an artificial intelligence server 700, any two designated accelerator cards in different accelerator card groups 301 are directly connected to the south. In an illustrative embodiment, because the two accelerator card groups 301 are in the same artificial intelligence server 700, the southbound direct connection can be a direct connection of the accelerator card through the corresponding southbound port, without the need to be connected via the southbound PCIe switch 302.

[0084] In the exemplary embodiment, a variety of feasible connection forms can be adopted between the southbound PCIe switch 302 and the acceleration card. For example, the southbound PCIe switch 302 and the acceleration card can be connected through at least one of an optical module and a copper cable.

[0085] The following is a description of an application scenario embodiment of an accelerator cluster supernode device and a computing acceleration device according to an embodiment of the present disclosure.

[0086] Application scenario example 1

[0087] Figure 8 is a structural diagram of an application scenario embodiment 1 of the present disclosure embodiment, such as Figure 8 As shown, application scenario embodiment 1 is a structure in which computing servers are interconnected via a southbound PCIe switch 302. Figure 8 As shown, the number of computing servers in this application scenario is four, namely, the first computing server 801, the second computing server 802, the third computing server 803, and the fourth computing server 804, each of which includes two artificial intelligence acceleration card groups 1 (equivalent to Figure 3 , Figure 7 Each AI acceleration card group 1 includes four acceleration cards, an internal PCIe switch (equivalent to Figure 4 , Figure 7 The northbound PCIe switch in the Figure 4 , Figure 7), where, for the sake of name distinction in the diagram, the four accelerator cards in the left artificial intelligence accelerator card group 1 in each computing server are named GPU0, GPU1, GPU2 and GPU3 respectively, and the four accelerator cards in the right artificial intelligence accelerator card 1 are named GPU4, GPU5, GPU6 and GPU7 respectively, the left CPU in each computing server is named CPU0, the right CPU in each computing server is named CPU1, the left northbound PCIe switch in each computing server is named PCIe SW0, and the right northbound PCIe switch in each computing server is named PCIe SW1. In addition, each computing server also includes a NIC card. It should be noted that, for the sake of simplicity in the diagram, the left and right artificial intelligence accelerator card groups 1 in each computing server are respectively connected to a NIC card, and the two NIC cards can be the same NIC card or two different NIC cards, that is, the computing server can use one NIC card or two NIC cards, and the same is true in subsequent embodiments.

[0088] The four accelerator cards in each artificial intelligence acceleration card group 1 are interconnected point-to-point. The four accelerator cards in each artificial intelligence acceleration card group 1 are coupled to their respective northbound PCIe switches. Each northbound PCIe switch is also coupled to the NIC card and the respective CPU. The two artificial intelligence acceleration card groups 1 on each computing server are interconnected through their respective CPUs.

[0089] Figure 8 In the application scenario embodiment 1 shown, a southbound PCIe switch 302 is included. The southbound PCIe switch 302 has 16 ports. According to computing requirements and / or design requirements, any two accelerator cards in each artificial intelligence accelerator card group 1 in each computing server are coupled to the southbound PCIe switch 302. A total of 16 accelerator cards can realize data interaction across artificial intelligence accelerator card groups 1 through the southbound PCIe switch 302. For example Figure 8 In each computing server, GPU0, GPU1, GPU6 and GPU7 are coupled to the southbound PCIe switch 302. Between the computing servers, GPU0, GPU1, GPU6 and GPU7 can realize data interaction across the artificial intelligence acceleration card group 1 through the southbound PCIe switch 302. The southbound PCIe switch 302 and each acceleration card can be connected through at least one of an optical module and a copper cable.

[0090] in addition, Figure 8In the application scenario embodiment 1 shown, there is no connection between the two artificial intelligence acceleration card groups 1 in each computing server, and the other four acceleration cards are connected to the southbound PCIe switch 302. One-to-one southbound connection across the artificial intelligence acceleration card group 1 can be performed inside the computing server. For example, GPU3 and GPU4 in each computing server are connected one-to-one southbound across the artificial intelligence acceleration card group 1, and GPU2 and GPU5 in each computing server are connected one-to-one southbound across the artificial intelligence acceleration card group 1. Because the one-to-one connection does not require the PCIe protocol to specify the acceleration card that receives data, there is no need to use a PCIe switch for connection, and only the relevant interconnection ports need to be used for direct connection.

[0091] In order to achieve the connection across the artificial intelligence acceleration card group 1, corresponding ports can be added to the acceleration card for the required southbound connection of the acceleration card.

[0092] Application scenario example 2

[0093] Fig. 9 is a structural diagram of an application scenario embodiment 2 of the present disclosure embodiment, and Figure 8 Compared with the application scenario embodiment 1 shown, the application scenario embodiment 2 adds a network switch 200. The network switch 200 can be an Ethernet switch. The network switch 200 is coupled to the NIC card in each computing server. The computing servers connected by the network switch 200 (for example, the four computing servers in the application scenario embodiment 2) and the network switch 200 together constitute a super computing node 400. The computing servers can realize northbound data interaction, upload data to the outside of the super computing node 400, and receive data from the super computing node 400 through the network switch 200.

[0094] Apart from this, the structures of other parts of application scenario embodiment 2 are the same as those of application scenario embodiment 1 and will not be repeated here.

[0095] Application scenario example 3

[0096] Fig.10 is a structural diagram of an application scenario embodiment 3 of the present disclosure embodiment, such as Fig.10 As shown, Figure 8 The application scenario shown in Example 1 and Fig. 9The difference between the application scenario embodiment 2 shown is that all the acceleration cards in the application scenario embodiment 3 are coupled to the southbound PCIe switch 302, wherein the southbound PCIe switch 302 includes two southbound PCIe sub-switches, namely a first southbound PCIe sub-switch 3021 and a second southbound PCIe sub-switch 3022, and the first southbound PCIe sub-switch 3021 and the second southbound PCIe sub-switch 3022 both have 16 ports, which can meet the southbound connection requirements of all the acceleration cards in the four computing servers.

[0097] The connection relationship between each accelerator card and the first southbound PCIe sub-switch 3021 and the second southbound PCIe sub-switch 3022 can be determined according to computing requirements and / or design requirements. Fig.10 In the illustrated embodiment, GPU0, GPU1, GPU6 and GPU7 in each computing server are coupled to the second southbound PCIe sub-switch 3022, and between the computing servers, GPU0, GPU1, GPU6 and GPU7 can realize data interaction across the artificial intelligence acceleration card group 1 through the second southbound PCIe sub-switch 3022; GPU2, GPU3, GPU4 and GPU5 in each computing server are coupled to the first southbound PCIe sub-switch 3021, and between the computing servers, GPU2, GPU3, GPU4 and GPU5 can realize data interaction across the artificial intelligence acceleration card group 1 through the first southbound PCIe sub-switch 3021.

[0098] In addition, the structures of other parts of application scenario embodiment 3 are the same as those of application scenario embodiments 1 and 2, and will not be repeated here.

[0099] Application scenario example 4

[0100] Fig.11 is a simplified structural diagram of the application scenario embodiment 4 of the embodiment of the present disclosure, such as Fig.11 As shown, the difference from the above-mentioned application scenario embodiments is that all the accelerator cards in application scenario embodiment 4 are coupled to the southbound PCIe switch 302, and each accelerator card can exchange southbound data with any other accelerator card in different artificial intelligence acceleration card groups 1 through the southbound PCIe switch 302.

[0101] Among them, the southbound PCIe switch 302 includes six southbound PCIe sub-switches, namely the first southbound PCIe sub-switch 3021, the second southbound PCIe sub-switch 3022, the third southbound PCIe sub-switch 3023, the fourth southbound PCIe sub-switch 3024, the fifth southbound PCIe sub-switch 3025 and the sixth southbound PCIe sub-switch 3026. Each southbound PCIe sub-switch has 16 ports, which can meet the southbound connection of all acceleration cards in the four computing servers.

[0102] Among them, the eight accelerator cards in the first computing server 801 and the eight accelerator cards in the second computing server 802 are connected through the first southbound PCIe sub-switch 3021, the eight accelerator cards in the first computing server 801 and the eight accelerator cards in the third computing server 803 are connected through the second southbound PCIe sub-switch 3022, the eight accelerator cards in the first computing server 801 and the eight accelerator cards in the fourth computing server 804 are connected through the third southbound PCIe sub-switch 3023, the eight accelerator cards in the second computing server 802 and the eight accelerator cards in the third computing server 803 are connected through the fourth southbound PCIe sub-switch 3024, the eight accelerator cards in the second computing server 802 and the eight accelerator cards in the fourth computing server 804 are connected through the fifth southbound PCIe sub-switch 3025, and the eight accelerator cards in the third computing server 803 and the eight accelerator cards in the fourth computing server 804 are connected through the sixth southbound PCIe sub-switch 3026. In order to distinguish and present more clearly in the diagram, Fig.11 All southbound connections are represented by dashed lines.

[0103] In addition, the structures of other parts of application scenario embodiment 4 are the same as those of application scenario embodiments 1 and 2, and will not be repeated here.

[0104] Application scenario example 5

[0105] Fig.12 is a structural diagram of an application scenario embodiment 5 of the present disclosure embodiment, such as Fig.12 As shown, the difference from the above-mentioned application scenario embodiments is that the application scenario embodiment 5 includes 16 computing servers, and the southbound PCIe switch 302 includes 8 16-port southbound PCIe sub-switches. For the sake of clarity, Fig.12 Only the connection relationship between the first computing server 801, the second computing server 802, the fifteenth computing server 815, the sixteenth computing server 816 and the eight southbound PCIe sub-switches is presented. The connection relationship between other computing servers not shown and the southbound PCIe sub-switches can be implemented with reference to the connection relationship shown.

[0106] like Fig.12 As shown, according to computing requirements and / or design requirements, in application scenario embodiment 5, GPU0 in each computing server is all coupled to the first southbound PCIe sub-switch 3021, so that southbound data interaction can be achieved between GPU0 in each computing server through the first southbound PCIe sub-switch 3021; ​​GPU1 in each computing server is all coupled to the second southbound PCIe sub-switch 3022, so that southbound data interaction can be achieved between GPU1 in each computing server through the second southbound PCIe sub-switch 3022; GPU2 in each computing server is all coupled to the third southbound PCIe sub-switch 3023, so that southbound data interaction can be achieved between GPU2 in each computing server through the third southbound PCIe sub-switch 3023; GPU3 in each computing server is all coupled to the fourth southbound PCIe sub-switch 3024, so that southbound data interaction can be achieved between GPU3 in each computing server through the fourth southbound PCIe sub-switch The GPUs 3024 in each computing server realize southbound data interaction; the GPUs 4 in each computing server are all coupled to the fifth southbound PCIe sub-switch 3025, so that the GPUs 4 in each computing server can realize southbound data interaction through the fifth southbound PCIe sub-switch 3025; the GPUs 5 in each computing server are all coupled to the sixth southbound PCIe sub-switch 3026, so that the GPUs 5 in each computing server can realize southbound data interaction through the sixth southbound PCIe sub-switch 3026; the GPUs 6 in each computing server are all coupled to the seventh southbound PCIe sub-switch 3027, so that the GPUs 6 in each computing server can realize southbound data interaction through the seventh southbound PCIe sub-switch 3027; the GPUs 7 in each computing server are all coupled to the eighth southbound PCIe sub-switch 3028, so that the GPUs 7 in each computing server can realize southbound data interaction through the eighth southbound PCIe sub-switch 3028. In order to distinguish between the diagrams and to present them more clearly, Fig.12 All southbound connections are represented by dashed lines.

[0107] In addition, the structures of other parts of application scenario embodiment 5 are the same as those of application scenario embodiments 1 and 2, and will not be repeated here.

[0108] Application scenario example 6

[0109] Fig.13 Schematic diagram of the structure of the application scenario embodiment 6 of the present disclosure embodiment, the application scenario embodiment 6 is a supercomputing cluster composed of multiple supercomputing nodes 400 interconnected. Fig.13As shown, the supercomputing cluster consists of several supercomputing nodes 400 connected via an IB / Ethernet switch 500 (equivalent to Figure 6 , Figure 7 The network switch 200 shown in the figure is interconnected, each supercomputing node 400 includes at least a plurality of computing servers 100 and a southbound PCIe switch 302, and the southbound PCIe switch 302 may further include a plurality of southbound PCIe sub-switches. Each supercomputing node 400 may adopt a different connection structure between the computing server 100 and the southbound PCIe switch 302 according to the design or computing task requirements. The connection interface may be, for example, the connection structure involved in the above-mentioned various application scenario embodiments or other achievable connection structures. Fig.13 The illustration is only for schematic purposes.

[0110] In addition, the structures of other parts of application scenario embodiment 6 are the same as those of application scenario embodiments 1 and 2, and will not be repeated here.

[0111] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. An accelerator cluster supernode device, characterized in that: include: A southbound PCIe switch and at least one artificial intelligence server; wherein, Each of the artificial intelligence servers includes at least two acceleration card groups, and each of the acceleration card groups contains at least one acceleration card; The southbound PCIe switch is coupled to the accelerator cards in at least two accelerator card groups in at least one of the artificial intelligence servers to form a southbound PCIe connection path between the accelerator cards across the accelerator card groups.

2. The accelerator cluster supernode device according to claim 1, characterized in that: When each of the acceleration card groups contains at least two acceleration cards, the southbound PCIe switch is coupled to at least one acceleration card in each of the acceleration card groups.

3. The accelerator cluster supernode device according to claim 1, characterized in that: Each of the artificial intelligence servers also includes: at least two processors, wherein in each of the artificial intelligence servers, the number of the processors is the same as the number of the acceleration card groups; and, At least two northbound PCIe switches, in each of the artificial intelligence servers, the number of the northbound PCIe switches is the same as the number of the acceleration card groups, each of the northbound PCIe switches is coupled one-to-one to each of the acceleration card groups, each of the northbound PCIe switches is coupled to at least one acceleration card in the coupled acceleration card group, and each of the northbound PCIe switches is coupled one-to-one to each of the processors to form a northbound PCIe connection path for the acceleration card group.

4. The accelerator cluster supernode device according to claim 3, characterized in that: In each of the artificial intelligence servers, the two acceleration card groups are northbound interconnected through the processors to which they are coupled.

5. The accelerator cluster supernode device according to claim 3, characterized in that: Each of the artificial intelligence servers also includes: NIC card, in each of the artificial intelligence servers, the NIC card is interconnected with the at least two northbound PCIe switches.

6. The accelerator cluster supernode device according to claim 5, characterized in that: The accelerator cluster super node device also includes: A network switch, wherein the network switch is interconnected with the NIC card in each of the artificial intelligence servers. When there are at least two artificial intelligence servers, the network switch forms a northbound PCIe connection path for the acceleration card group via the NIC card and the network switch between at least two of the artificial intelligence servers.

7. The accelerator cluster supernode device according to claim 1, characterized in that: The southbound PCIe switch comprises: At least one southbound PCIe sub-switch, the total number of ports of all the southbound PCIe sub-switches being not less than the number of ports of the acceleration card coupled to the southbound PCIe switch in the accelerator cluster supernode device.

8. The accelerator cluster supernode device according to claim 1, characterized in that: The acceleration card groups are paired in pairs to form a computing acceleration unit in an artificial intelligence server.

9. The accelerator cluster supernode device according to claim 8, characterized in that: Between two acceleration card groups in one of the artificial intelligence servers, any two designated acceleration cards in different acceleration card groups are directly connected to the south.

10. The accelerator cluster supernode device according to claim 1, characterized in that: The southbound PCIe switch and the acceleration card are connected via at least one of an optical module and a copper cable.

11. A computing acceleration device, characterized in that: include: At least two acceleration card groups, each of which contains at least one acceleration card; and A southbound PCIe switch is coupled to the acceleration cards in the at least two acceleration card groups to form a southbound PCIe connection path between the acceleration cards and across the acceleration card groups.

Images