Artificial intelligence cluster networking topology method and device, electronic equipment and storage medium

Abstract

The present application relates to the technical field of data center network, and particularly relates to a kind of intelligent computing cluster networking topology method and device, electronic equipment and storage medium, wherein, method includes: determining the total quantity of GPU in the to-be-networked intelligent computing cluster and the quantity of GPU connected by each switch;Identify the current network port type of GPU, and determine the switch quantity of each layer in double-layer topology structure according to the current network port type of GPU, the total quantity of GPU and the quantity of GPU connected by each switch;Determine the interconnection relationship between each GPU and multiple switches based on the current network port type of GPU, and interconnect GPU and switch in the to-be-networked intelligent computing cluster according to the interconnection relationship between each GPU and multiple switches, obtain the intelligent computing cluster networking topology representation result, thereby, solve the problem that related art is prone to cause training efficiency to decline and topology structure robustness to be poor under large-scale All-to-all traffic due to uneven load, improve the performance of large model training and inference, improve the robustness of topology structure.

Description

Technical Field

[0001] This invention relates to the field of data center network technology, and in particular to a method, apparatus, electronic device and storage medium for intelligent computing cluster networking topology. Background Technology

[0002] With the rapid development of generative artificial intelligence, the scale of model parameters and the computational load of training and inference are growing exponentially, driving the continuous expansion of computing clusters and a significant increase in computing costs. Since a single GPU's memory cannot accommodate all the parameters of a large model, training requires scheduling a large number of GPUs and relying on various parallel strategies. Data synchronization between different GPUs requires ensemble communication operations. All-reduce is widely used in data parallelism and tensor parallelism scenarios. With the rise of Hybrid Expert (MoE) models, all-to-all communication, which expert parallelism relies on, has also become crucial. GPU interconnect networks are divided into scale-up networks implemented within a single machine via NVLink or PCIe, and scale-out networks built across servers using Ethernet, InfiniBand (IB), or RoCE. Their topology design directly determines ensemble communication performance and the efficiency of large model training; therefore, cost-effective and efficient network topology design has become a key research focus in the industry.

[0003] In related technologies, the network topology of intelligent computing clusters is usually the Rail-Optimized Fat-Tree method. Through the rail switch, GPUs with the same ID in different servers are interconnected within a single hop range, and data interaction can be completed without multi-hop forwarding. This improves the performance of all-reduce ensemble communication and provides efficiency support for data synchronization in large model training.

[0004] However, the guide rail optimization method for fat tree topology has obvious limitations. On the one hand, the optimization of the guide rail optimization method for fat tree topology only focuses on the all-reduce communication operation and fails to cover the all-to-all communication that the training of hybrid expert (MoE) models depends on. Under large-scale all-to-all communication traffic, the load unevenness problem is prone to occur, resulting in a decrease in training efficiency. On the other hand, when a single point of failure occurs in the network, it will directly cause large-scale GPU communication to be blocked, which cannot guarantee the continuity of communication and thus cause serious degradation of the overall performance of the cluster. It is difficult to meet the stability requirements of large model training, which urgently needs to be solved. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for intelligent computing cluster networking topology, in order to solve the problems in related technologies where training efficiency is easily reduced and topology robustness is poor due to uneven load under large-scale all-to-all traffic, thereby improving the performance of large model training and inference and enhancing the robustness of the topology.

[0006] A first aspect of the present invention provides a method for forming a network topology of an intelligent computing cluster, comprising the following steps: determining the total number of GPUs in the intelligent computing cluster to be networked and the number of GPUs connected to each switch; identifying the current network port type of the GPUs, and determining the number of switches in each layer of the two-layer topology based on the current network port type of the GPUs, the total number of GPUs, and the number of GPUs connected to each switch; determining the interconnection relationship between each GPU and multiple switches based on the current network port type of the GPUs, and interconnecting the GPUs and switches in the intelligent computing cluster to be networked according to the interconnection relationship between each GPU and multiple switches, thereby obtaining the intelligent computing cluster network topology representation result.

[0007] Furthermore, in some embodiments, the current network interface type of the GPU is a single-port network card or a dual-port network card.

[0008] Further, in some embodiments, the current network port type of the GPU is the single-port network card type. The interconnection relationship between each GPU and multiple switches is determined based on the current network port type of the GPU, and the GPUs and switches in the intelligent computing cluster to be networked are interconnected according to the interconnection relationship between each GPU and multiple switches to obtain the intelligent computing cluster network topology representation result. This includes: numbering the network card of each GPU based on the total number of GPUs, and classifying all GPUs into odd-numbered network cards and even-numbered network cards based on parity classification rules; determining the Blo of each layer of the topology. The number of ck is determined, and the number of switches and network card ports in each block is determined based on the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each topology layer. When the network card of the GPU is an even-numbered network card, the number of switches and network card ports in each block are topologically linked based on a preset first interconnection relationship. When the network card of the GPU is an odd-numbered network card, the number of switches and network card ports in each block are topologically linked based on a preset second interconnection relationship, thereby obtaining the network topology representation result of the intelligent computing cluster.

[0009] Further, in some embodiments, the preset first interconnection relationship is: the p-th network card port is connected to the [(p-1)mod[n / (2*k*b1)]]+1-th switch within the same block, 1≤p≤n / (2*b1), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b1 is the number of blocks in the first layer topology; the preset second interconnection relationship is: the q-th network card port is connected to the [(q-1)mod[n / (2*k*b2)]]+1-th switch within the same block, 1≤q≤n / (2*b2), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second layer topology.

[0010] Further, in some embodiments, the current network port type of the GPU is the dual-port network card type. The process of determining the interconnection relationship between each GPU and multiple switches based on the current network port type of the GPU, and interconnecting the GPUs and switches in the intelligent computing cluster to be networked according to the interconnection relationship between each GPU and multiple switches, to obtain the intelligent computing cluster network topology representation result, includes: determining the number of blocks in each topology layer, and determining the number of switches and network card ports in each block based on the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each topology layer; when the network card port of the current GPU is the first port, performing topological linking on the number of switches and network card ports in each block within the first topology layer based on a preset third interconnection relationship; when the network card port of the current GPU is the second port, performing topological linking on the number of switches and network card ports in each block within the second topology layer based on the preset fourth interconnection relationship, to obtain the intelligent computing cluster network topology representation result.

[0011] Furthermore, in some embodiments, the preset third interconnection relationship is: the xth network card port is connected to the [(x-1)mod[n / (k*b1)]]+1th switch within the same block, 1≤x≤n / b1, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b1 is the number of blocks in the first layer topology; the preset fourth interconnection relationship is: the yth network card port is connected to the [(y-1)mod[n / (k*b2)]]+1th switch within the same block, 1≤y≤n / b2, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second layer topology.

[0012] Furthermore, in some embodiments, the two switches in the dual-layer topology are connected in a fully bipartite graph manner.

[0013] The intelligent computing cluster networking topology method provided by the present invention first determines the total number of GPUs in the intelligent computing cluster and the number of GPUs connected to a single switch, identifies the GPU network port type, and determines the number of switches in each layer of the two-layer topology accordingly. Finally, it determines the interconnection relationship between GPUs and switches based on the GPU network port type and connects them to obtain the intelligent computing cluster networking topology representation result. This solves the problem in related technologies that training efficiency is easily reduced and topology robustness is poor under large-scale all-to-all traffic due to uneven load. It improves the performance of large model training and inference and enhances the robustness of the topology.

[0014] A second aspect of the present invention provides a smart computing cluster networking topology device, the device comprising: a determining module, configured to determine the total number of GPUs in the smart computing cluster to be networked and the number of GPUs connected to each switch; a calculating module, configured to identify the current network port type of the GPUs, and determine the number of switches in each layer of a two-layer topology based on the current network port type of the GPUs, the total number of GPUs, and the number of GPUs connected to each switch; and a generating module, configured to determine the interconnection relationship between each GPU and multiple switches based on the current network port type of the GPUs, and interconnect the GPUs and switches in the smart computing cluster to be networked based on the interconnection relationship between each GPU and multiple switches, thereby obtaining a smart computing cluster networking topology representation result.

[0015] Furthermore, in some embodiments, the current network interface type of the GPU is a single-port network card or a dual-port network card.

[0016] Further, in some embodiments, the current network interface type of the GPU is the single-port network card type. The generation module is specifically used for: numbering the network card of each GPU based on the total number of GPUs, and classifying all GPUs into odd-numbered network cards and even-numbered network cards according to the parity classification rule; determining the number of blocks in each layer of the topology, and determining the number of switches and network card ports in each block according to the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each layer of the topology; when the network card of the GPU is an even-numbered network card, performing topological linking of the number of switches and network card ports in each block based on a preset first interconnection relationship; when the network card of the GPU is an odd-numbered network card, performing topological linking of the number of switches and network card ports in each block based on a preset second interconnection relationship, thereby obtaining the network topology representation result of the intelligent computing cluster.

[0017] Further, in some embodiments, the preset first interconnection relationship is: the p-th network card port is connected to the [(p-1)mod[n / (2*k*b1)]]+1-th switch within the same block, 1≤p≤n / (2*b1), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b1 is the number of blocks in the first layer topology; the preset second interconnection relationship is: the q-th network card port is connected to the [(q-1)mod[n / (2*k*b2)]]+1-th switch within the same block, 1≤q≤n / (2*b2), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second layer topology.

[0018] Furthermore, in some embodiments, the current network port type of the GPU is the dual-port network card type. The generation module is further configured to: determine the number of blocks in each layer of the topology, and determine the number of switches and network card ports in each block based on the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each layer of the topology; when the network card port of the current GPU is the first port, perform topological linking on the number of switches and network card ports in each block of the first layer of the topology based on a preset third interconnection relationship; when the network card port of the current GPU is the second port, perform topological linking on the number of switches and network card ports in each block of the second layer of the topology based on the preset fourth interconnection relationship, thereby obtaining the intelligent computing cluster network topology representation result.

[0019] Furthermore, in some embodiments, the preset third interconnection relationship is: the xth network card port is connected to the [(x-1)mod[n / (k*b1)]]+1th switch within the same block, 1≤x≤n / b1, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b1 is the number of blocks in the first layer topology; the preset fourth interconnection relationship is: the yth network card port is connected to the [(y-1)mod[n / (k*b2)]]+1th switch within the same block, 1≤y≤n / b2, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second layer topology.

[0020] Furthermore, in some embodiments, the two switches in the dual-layer topology are connected in a fully bipartite graph manner.

[0021] The intelligent computing cluster networking topology device provided in this embodiment of the invention first determines the total number of GPUs in the intelligent computing cluster and the number of GPUs connected to a single switch, identifies the GPU network port type, and determines the number of switches in each layer of the two-layer topology accordingly. Finally, it determines the interconnection relationship between the GPUs and the switches based on the GPU network port type and connects them to obtain the intelligent computing cluster networking topology representation result. This solves the problem in related technologies that training efficiency is easily reduced and topology robustness is poor under large-scale all-to-all traffic due to uneven load. It improves the performance of large model training and inference and enhances the robustness of the topology.

[0022] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the intelligent computing cluster networking topology method described in the above embodiments.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the intelligent computing cluster networking topology method as described in the above embodiments.

[0024] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0025] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0026] Figure 1 A flowchart of a smart computing cluster networking topology method provided according to an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of a single-card single-port configuration topology (4096, 64, 32, 1) according to a specific embodiment of the present invention;

[0028] Figure 3 A schematic diagram of a single-card single-port configuration topology (4096,64,8,8) provided according to a specific embodiment of the present invention;

[0029] Figure 4 A schematic diagram of a single-card single-port configuration topology (8192,64,64,1) provided according to a specific embodiment of the present invention;

[0030] Figure 5 A schematic diagram of a single-card single-port configuration topology (8192, 64, 16, 16) according to a specific embodiment of the present invention;

[0031] Figure 6 A schematic diagram of a single-card dual-port configuration topology (4096, 128, 32, 1) provided according to a specific embodiment of the present invention;

[0032] Figure 7 A schematic diagram of a single-card dual-port configuration topology (4096,128,4,4) provided according to a specific embodiment of the present invention;

[0033] Figure 8 A schematic diagram of a single-card dual-port configuration topology (16384, 128, 128, 1) according to a specific embodiment of the present invention;

[0034] Figure 9 A schematic diagram of a single-card dual-network port configuration topology (16384,128,16,16) provided according to a specific embodiment of the present invention;

[0035] Figure 10 A schematic diagram of a single-card dual-port configuration topology (16,4,4,1) provided according to a specific embodiment of the present invention;

[0036] Figure 11 This is a block diagram of a smart computing cluster networking topology device provided according to an embodiment of the present invention;

[0037] Figure 12 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0038] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0039] The following description, with reference to the accompanying drawings, outlines an intelligent computing cluster networking topology method, apparatus, electronic device, and storage medium according to embodiments of the present invention. Addressing the issues mentioned in the background art where uneven load distribution under large-scale all-to-all traffic conditions easily leads to decreased training efficiency and poor topology robustness, the present invention provides an intelligent computing cluster networking topology method. First, it determines the total number of GPUs in the intelligent computing cluster and the number of GPUs connected to a single switch. It then identifies the GPU network port types and determines the number of switches in each layer of the two-layer topology accordingly. Finally, it determines the interconnection relationship between GPUs and switches based on the GPU network port types and establishes connections, obtaining the intelligent computing cluster networking topology representation result. This method solves the problem in related technologies where uneven load distribution under large-scale all-to-all traffic conditions easily leads to decreased training efficiency and poor topology robustness, improving the performance of large model training and inference, and enhancing the robustness of the topology.

[0040] Specifically, Figure 1 This is a flowchart of a smart computing cluster networking topology method provided according to an embodiment of the present invention.

[0041] like Figure 1 As shown, the intelligent computing cluster networking topology method includes the following steps:

[0042] In step S101, the total number of GPUs in the intelligent computing cluster to be networked and the number of GPUs connected to each switch are determined.

[0043] The total number of GPUs in the intelligent computing cluster to be networked refers to the total number of all GPUs in the intelligent computing cluster networking topology task, and the number of GPUs connected to each switch refers to the number of GPU devices that a single network switch can actually directly or indirectly connect to in the intelligent computing cluster networking.

[0044] Specifically, in this embodiment of the invention, the total number of GPUs in the intelligent computing cluster to be networked is first determined based on communication requirements and large model parameter requirements, and the number of GPUs connected to each switch is determined based on the specific parameters of the switch (e.g., communication bandwidth, number of switch ports).

[0045] In step S102, the current network port type of the GPU is identified, and the number of switches in each layer of the two-layer topology is determined based on the current network port type of the GPU, the total number of GPUs, and the number of GPUs connected to each switch.

[0046] In some embodiments, the GPU's current network interface type is either a single-port network card or a dual-port network card.

[0047] Specifically, let the total number of GPUs in the intelligent computing cluster to be networked be n, and the number of GPUs connected to each switch be k. If the current network interface type of the GPU is a single-port network card, then the number of switches at each layer is n / (2*k). If the current network interface type of the GPU is a dual-port network card, then the number of switches at each layer is n / k.

[0048] For example, suppose the total number of GPUs in the intelligent computing cluster to be networked is 4096, and the number of GPUs connected to each switch is 64. If the current network port type of the GPU is a single-port network card, then the number of switches at each layer is 4096 / (2*64)=32. If the current network port type of the GPU is a dual-port network card, then the number of switches at each layer is 4096 / 64=64.

[0049] In step S103, the interconnection relationship between each GPU and multiple switches is determined based on the current network port type of the GPU, and the GPUs and switches in the intelligent computing cluster to be networked are interconnected according to the interconnection relationship between each GPU and multiple switches to obtain the network topology representation result of the intelligent computing cluster.

[0050] In some embodiments, the current network interface type of the GPU is a single-port network card. Based on the current network interface type of the GPU, the interconnection relationship between each GPU and multiple switches is determined. The GPUs and switches in the intelligent computing cluster to be networked are then interconnected according to the interconnection relationship between each GPU and multiple switches to obtain the intelligent computing cluster network topology representation result. This includes: numbering the network card of each GPU based on the total number of GPUs, and classifying all GPUs into odd-numbered and even-numbered network cards based on parity classification rules; determining the number of blocks in each topology layer, and determining the number of switches and network card ports within each block based on the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each topology layer; when the network card of the GPU is an even-numbered network card, topological linking is performed on the number of switches and network card ports within each block based on a preset first interconnection relationship; when the network card of the GPU is an odd-numbered network card, topological linking is performed on the number of switches and network card ports within each block based on a preset second interconnection relationship to obtain the intelligent computing cluster network topology representation result.

[0051] The number of blocks in each topology layer refers to the number of blocks into which switches on the same layer are divided according to modularity.

[0052] Specifically, let the total number of GPUs in the intelligent computing cluster to be networked be n, the number of GPUs connected to each switch be k, the number of blocks in the first layer of the two-layer topology be b1, and the number of blocks in the second layer of the two-layer topology be b2. First, the network cards configured for each GPU are numbered from 0 to n-1. These numbers are then divided into odd-numbered and even-numbered network cards. Even-numbered network cards are connected to the switches in the first layer of the two-layer topology, and odd-numbered network cards are connected to the switches in the second layer of the two-layer topology. Each block contains n / (2*k*b1) switches and n / (2*b1) network interface cards (NICs). Based on a preset first interconnection relationship, the number of switches and NICs in each block are topologically linked. When the GPU's NICs are odd-numbered, each block contains n / (2*k*b2) switches and n / (2*b2) NICs. Based on a preset second interconnection relationship, the number of switches and NICs in each block are topologically linked, ultimately yielding the intelligent computing cluster network topology representation result.

[0053] For example, suppose the total number of GPUs in the intelligent computing cluster to be networked is 4096, each switch connects to 64 GPUs, the first layer of the two-layer topology has 32 blocks, and the second layer has 1 block. Each GPU's network interface card (NIC) is numbered from 0 to 4095. These numbers are then categorized into odd-numbered and even-numbered NICs, with even-numbered NICs being (0, 2, 4, ..., 4094) and odd-numbered NICs being (1, 3, 5, ..., 4095). Each block contains 4096 GPUs. / (2*64*32)=1 switch and 4096 / (2*32)=64 network interface card (NIC) ports. Then, based on the preset first interconnection relationship, the number of switches and NIC ports in each block are topologically linked. When the GPU's NICs are odd-numbered, each block will have 4096 / (2*64*1)=32 switches and 4096 / 2=2048 NIC ports. Based on the preset second interconnection relationship, the number of switches and NIC ports in each block are topologically linked, and finally, the intelligent computing cluster network topology representation result is obtained.

[0054] In some embodiments, the preset first interconnection relationship is: the p-th network card port is connected to the [(p-1)mod[n / (2*k*b1)]]+1-th switch within the same block, 1≤p≤n / (2*b1), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b1 is the number of blocks in the first layer topology; the preset second interconnection relationship is: the q-th network card port is connected to the [(q-1)mod[n / (2*k*b2)]]+1-th switch within the same block, 1≤q≤n / (2*b2), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second layer topology.

[0055] For example, Figure 2 This is a schematic diagram of a single-card, single-port configuration topology (4096, 64, 32, 1) according to a specific embodiment of the present invention, as shown below. Figure 2As shown, the total number of GPUs in the intelligent computing cluster to be networked is 4096, each switch connects to 64 GPUs, the first layer of the two-layer topology has 32 blocks, the second layer has 1 block, Switch2 and Switch64 are the first layer switches, Switch1, Switch3, ..., Switch63 are the second layer switches, and Server is a server equipped with 8 single-port network interface GPUs; based on The pre-defined first interconnection relationship establishes a topology link for the number of switches and network interface cards (NICs) within each block. Specifically, the first NIC port connects to the [(1-1)mod[4096 / (2*64*32)]]+1=1th switch within the same block, the second even-numbered NIC port connects to the [(2-1)mod[4096 / (2*64*32)]]+1=1th switch within the same block, and the second NIC port connects to the [(2-1)mod[4096 / (2*64*32)]]+1=1th switch within the same block. *64*32)]]+1=1 switch, the 64th network card port is connected to the [(64-1)mod[4096 / (2*64*32)]]+1=1th switch in the same block, where 1≤p≤64; based on the preset second interconnection relationship, the number of switches and network card ports in each block are topologically linked, that is, the 1st network card port is connected to the [(1-1)mod[4096 / (2*64*1)]]+1=1th switch in the same block, the 2nd network card port is connected to The 2048th NIC port is connected to the 2nd switch in the same block, and the 3rd NIC port is connected to the 3rd switch in the same block. The 2048th NIC port is connected to the 32nd switch in the same block, where 1 ≤ q ≤ 2048. Thus, all ports of the switch are interconnected with the single-port NIC.

[0056] It should be understood that, Figure 3 In the illustrated embodiment, the first layer of the dual-layer topology has 32 switches, with one switch in each block. Each switch is connected to 64 GPUs in the same block. The second layer of the dual-layer topology also has 32 switches and 2048 network interface card (NIC) ports. Therefore, each switch in the second layer of the dual-layer topology has 64 NIC ports connected to the GPUs.

[0057] Furthermore, Figure 3This is a schematic diagram of a single-card, single-port configuration topology (4096,64,8,8) according to a specific embodiment of the present invention. Figure 4 This is a schematic diagram of a single-card, single-port configuration topology (8192,64,64,1) according to a specific embodiment of the present invention. Figure 5 This is a schematic diagram of a single-card, single-network-port configuration topology (8192,64,16,16) according to a specific embodiment of the present invention. It should be noted that... Figure 3 , Figure 4 and Figure 5 Interconnection methods and Figure 2 The interconnection methods shown are the same and can be derived from the first interconnection relationship and the second interconnection relationship, which will not be elaborated here.

[0058] Furthermore, in some embodiments, the current network port type of the GPU is a dual-port network card type. Based on the current network port type of the GPU, the interconnection relationship between each GPU and multiple switches is determined, and the GPUs and switches in the intelligent computing cluster to be networked are interconnected according to the interconnection relationship between each GPU and multiple switches to obtain the intelligent computing cluster network topology representation result, including: determining the number of blocks in each layer of the topology structure, and determining the number of switches and network card ports in each block according to the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each layer of the topology structure; when the network card port of the current GPU is the first port, the number of switches and network card ports in each block in the first layer of the topology structure are topologically linked based on the preset third interconnection relationship; when the network card port of the current GPU is the second port, the number of switches and network card ports in each block in the second layer of the topology structure are topologically linked based on the preset fourth interconnection relationship to obtain the intelligent computing cluster network topology representation result.

[0059] Specifically, let the total number of GPUs in the intelligent computing cluster to be networked be n, the number of GPUs connected to each switch be k, the number of blocks in the first layer of the two-layer topology be b1, the number of blocks in the second layer of the two-layer topology be b2, and the two ports of the dual-port network card will be connected to switches of different layers respectively, that is, the first port is connected to the first layer switch, and the second port is connected to the second layer switch. When the first port is interconnected with the first layer switch, there will be n / (k*b1) switches and n / b1 network card ports in each block. Based on the preset third interconnection relationship, the number of switches and network card ports in each block of the first layer topology are topologically linked. When the second port is interconnected with the second layer switch, there will be n / (k*b2) switches and n / b2 network card ports in each block. Based on the preset fourth interconnection relationship, the number of switches and network card ports in each block of the second layer topology are topologically linked, thus obtaining the network topology representation result of the intelligent computing cluster.

[0060] For example, suppose the total number of GPUs in the intelligent computing cluster to be networked is 4096, each switch connects to 128 GPUs, the first layer of the two-layer topology has 32 blocks, and the second layer has 1 block. For the interconnection between the first port and the first-layer switch, each block will have 4096 / (128*32)=1 switch and 4096 / 32=128 network interface cards (NICs). Based on the preset third interconnection relationship, the number of switches and NICs in each block of the first-layer topology is topologically linked. For the interconnection between the second port and the second-layer switch, each block will have 4096 / 128=32 switches and 4096 / 1=4096 NICs. Based on the preset fourth interconnection relationship, the number of switches and NICs in each block of the second-layer topology is topologically linked, and the final intelligent computing cluster network topology representation result is obtained.

[0061] In some embodiments, the preset third interconnection relationship is: the xth network card port is connected to the [(x-1)mod[n / (k*b1)]]+1th switch within the same block, 1≤x≤n / b1, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b1 is the number of blocks in the first layer topology; the preset fourth interconnection relationship is: the yth network card port is connected to the [(y-1)mod[n / (k*b2)]]+1th switch within the same block, 1≤y≤n / b2, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second layer topology.

[0062] For example, Figure 6 This is a schematic diagram of a single-card dual-port configuration topology (4096,128,32,1) according to a specific embodiment of the present invention, as shown below. Figure 6 As shown, the total number of GPUs in the intelligent computing cluster to be networked is 4096, each switch connects to 128 GPUs, the first layer of the two-layer topology has 32 blocks, the second layer has 1 block, Switch2 and Switch64 are the first layer switches, Switch1, Switch15, ..., Switch63 are the second layer switches, and Server is a server equipped with 8 dual-port network interface cards (NICs) for GPUs; based on The pre-defined third interconnection relationship establishes a topology link for the number of switches and network interface card (NIC) ports within each block. For the interconnection of the first port of a dual-port NIC GPU with a Layer 1 switch, the first NIC port connects to the [(1-1)mod[4096 / (128*32)]]+1=1th switch within the same block, the second NIC port connects to the [(2-1)mod[4096 / (128*32)]]+1=1th switch within the same block, and the 128th NIC port connects to the [(2-1)mod[4096 / (128*32)]]+1=1th switch within the same block. The [(128-1)mod[4096 / (128*32)]]+1=1th switch within k, where 1≤x≤128; For the interconnection between the second port of a dual-port NIC GPU and a Layer 2 switch, the first NIC port connects to the [(1-1)mod[4096 / (128*1)]]+1=1th switch within the same block, the second NIC port connects to the [(2-1)mod[4096 / (128*1)]]+1=2th switch within the same block, and the third NIC port... The network interface card (NIC) is connected to the 3rd switch in the same block (3-1)mod[4096 / (128*1)]]+1=3rd switch, and the 4096th NIC port is connected to the 32nd switch in the same block (4096-1)mod[4096 / (128*1)]]+1=32nd switch, where 1≤y≤4096. Thus, all ports of the switch are interconnected with the dual-port NIC.

[0063] It should be noted that, Figure 6The second layer of the two-layer topology shown contains one block with 32 switches. The first network interface card (NIC) port is connected to the first switch in the same block, the 32nd NIC port is connected to the 32nd switch in the same block, and the 33rd NIC port is connected to the first switch in the same block. That is, the interconnection between the NIC ports and the switches is in a 32-bit loop. Each switch is connected to 128 GPUs to achieve the interconnection between the NIC ports and the switches.

[0064] Furthermore, Figure 7 This is a schematic diagram of a single-card dual-port configuration topology (4096,128,4,4) according to a specific embodiment of the present invention. Figure 8 This is a schematic diagram of a single-card dual-port configuration topology (16384,128,128,1) according to a specific embodiment of the present invention. Figure 9 This is a schematic diagram of a single-card dual-port configuration topology (16384,128,16,16) according to a specific embodiment of the present invention. It should be noted that... Figure 7 , Figure 8 and Figure 9 Interconnection methods and Figure 6 The interconnection methods shown are the same and can be derived from the third and fourth interconnection relationships, which will not be elaborated here.

[0065] Furthermore, in some embodiments, the two switches in the dual-layer topology are connected in a fully bipartite graph manner.

[0066] Specifically, in this embodiment of the invention, the two-layer switches in the dual-layer topology are connected in a completely bipartite graph, solving the problems of traditional topologies through full-link coverage, such as... Figure 2 As shown, any GPU communication requires a maximum of only 2 hops, which is better than the 5 hops of a three-layer fat tree, significantly improving the efficiency of All-reduce / All-to-all communication. Furthermore, even if a single point of failure occurs, communication with the Layer 2 switch can still be achieved through other network interface card ports. Figure 2 The middle two layers consist of 64 switches, compared to 96 switches in the three-layer fat tree method, reducing hardware costs by one-third and significantly lowering design costs.

[0067] To enable those skilled in the art to better understand the intelligent computing cluster networking topology method of the present invention, the following explanation will be provided in conjunction with specific embodiments.

[0068] Figure 10 A schematic diagram of a single-card dual-port configuration topology (16,4,4,1) according to a specific embodiment of the present invention is shown below. Figure 10As shown, a single-card dual-port server configuration (16, 4, 4, 1) is constructed using 4 servers and 8 Mellanox QM9790IB switches. Each server contains 4 NVIDIA H800 GPUs, and the GPUs on the same server are interconnected via 200GB / s NVLink. A rail-optimized fat-tree topology was constructed using 16 NVIDIA ConnectX-7400GbE network cards, and the same topology was constructed using 16 NVIDIA ConnectX-7400GbE network cards. Tests were conducted using the NCCL2.21.5 aggregated communication library. Compared to the rail-optimized fat-tree topology, this topology reduces network hardware costs by one-third while achieving the same All-reduce and All-to-all communication performance, demonstrating the effectiveness of this embodiment.

[0069] The intelligent computing cluster networking topology method provided by the present invention first determines the total number of GPUs in the intelligent computing cluster and the number of GPUs connected to a single switch, identifies the GPU network port type, and determines the number of switches in each layer of the two-layer topology accordingly. Finally, it determines the interconnection relationship between GPUs and switches based on the GPU network port type and connects them to obtain the intelligent computing cluster networking topology representation result. This solves the problem in related technologies that training efficiency is easily reduced and topology robustness is poor under large-scale all-to-all traffic due to uneven load. It improves the performance of large model training and inference and enhances the robustness of the topology.

[0070] Next, the intelligent computing cluster networking topology device provided according to an embodiment of the present invention is described with reference to the accompanying drawings.

[0071] Figure 11 This is a block diagram of a smart computing cluster networking topology device provided according to an embodiment of the present invention.

[0072] like Figure 11 As shown, the intelligent computing cluster networking topology device 10 includes: a determination module 100, a computing module 200, and a generation module 300.

[0073] The determination module 100 is used to determine the total number of GPUs in the intelligent computing cluster to be networked and the number of GPUs connected to each switch; the calculation module 200 is used to identify the current network port type of the GPUs and determine the number of switches in each layer of the two-layer topology based on the current network port type of the GPUs, the total number of GPUs, and the number of GPUs connected to each switch; the generation module 300 determines the interconnection relationship between each GPU and multiple switches based on the current network port type of the GPUs, and interconnects the GPUs and switches in the intelligent computing cluster to be networked according to the interconnection relationship between each GPU and multiple switches to obtain the intelligent computing cluster network topology representation result.

[0074] Furthermore, in some embodiments, the GPU's current network interface type is either a single-port network card or a dual-port network card.

[0075] Furthermore, in some embodiments, the current network interface type of the GPU is a single-port network card. The generation module 300 is specifically used for: numbering the network card of each GPU based on the total number of GPUs, and classifying all GPUs into odd-numbered network cards and even-numbered network cards based on parity classification rules; determining the number of blocks in each topology layer, and determining the number of switches and network card ports in each block based on the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each topology layer; when the network card of the GPU is an even-numbered network card, performing topological linking of the number of switches and network card ports in each block based on a preset first interconnection relationship; when the network card of the GPU is an odd-numbered network card, performing topological linking of the number of switches and network card ports in each block based on a preset second interconnection relationship, thereby obtaining the intelligent computing cluster network topology representation result.

[0076] Furthermore, in some embodiments, the preset first interconnection relationship is: the p-th network card port is connected to the [(p-1)mod[n / (2*k*b1)]]+1-th switch within the same block, 1≤p≤n / (2*b1), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b1 is the number of blocks in the first layer topology; the preset second interconnection relationship is: the q-th network card port is connected to the [(q-1)mod[n / (2*k*b2)]]+1-th switch within the same block, 1≤q≤n / (2*b2), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second layer topology.

[0077] Furthermore, in some embodiments, the current network interface type of the GPU is a dual-port network card type. The generation module 300 is also used to: determine the number of blocks in each layer of the topology, and determine the number of switches and network card ports in each block based on the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each layer of the topology; when the network card port of the current GPU is the first port, perform topological linking on the number of switches and network card ports in each block in the first layer of the topology based on a preset third interconnection relationship; when the network card port of the current GPU is the second port, perform topological linking on the number of switches and network card ports in each block in the second layer of the topology based on a preset fourth interconnection relationship, thereby obtaining the intelligent computing cluster networking topology representation result.

[0078] Furthermore, in some embodiments, the preset third interconnection relationship is: the xth network card port is connected to the [(x-1)mod[n / (k*b1)]]+1th switch within the same block, 1≤x≤n / b1, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b1 is the number of blocks in the first layer topology; the preset fourth interconnection relationship is: the yth network card port is connected to the [(y-1)mod[n / (k*b2)]]+1th switch within the same block, 1≤y≤n / b2, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second layer topology.

[0079] Furthermore, in some embodiments, the two switches in the dual-layer topology are connected in a fully bipartite graph manner.

[0080] It should be noted that the above explanation of the intelligent computing cluster networking topology method embodiment also applies to the intelligent computing cluster networking topology device of this embodiment, and will not be repeated here.

[0081] The intelligent computing cluster networking topology device provided in this embodiment of the invention first determines the total number of GPUs in the intelligent computing cluster and the number of GPUs connected to a single switch, identifies the GPU network port type, and determines the number of switches in each layer of the two-layer topology accordingly. Finally, it determines the interconnection relationship between the GPUs and the switches based on the GPU network port type and connects them to obtain the intelligent computing cluster networking topology representation result. This solves the problem in related technologies that training efficiency is easily reduced and topology robustness is poor under large-scale all-to-all traffic due to uneven load. It improves the performance of large model training and inference and enhances the robustness of the topology.

[0082] Figure 12 This is a schematic diagram of an electronic device provided according to an embodiment of the present invention. The electronic device may include:

[0083] The memory 1201, the processor 1202, and the computer program stored on the memory 1201 and executable on the processor 1202.

[0084] When the processor 1202 executes the program, it implements the intelligent computing cluster networking topology method provided in the above embodiments.

[0085] Furthermore, the electronic device also includes:

[0086] Communication interface 1203 is used for communication between memory 1201 and processor 1202.

[0087] The memory 1201 is used to store computer programs that can run on the processor 1202.

[0088] The memory 1201 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage.

[0089] If the memory 1201, processor 1202, and communication interface 1203 are implemented independently, then the communication interface 1203, memory 1201, and processor 1202 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0090] Optionally, in a specific implementation, if the memory 1201, processor 1202, and communication interface 1203 are integrated on a single chip, then the memory 1201, processor 1202, and communication interface 1203 can communicate with each other through an internal interface.

[0091] The processor 1202 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0092] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described intelligent computing cluster networking topology method.

[0093] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0094] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0095] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0096] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0097] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A method for forming a network topology for an intelligent computing cluster, characterized in that, Includes the following steps: Determine the total number of GPUs in the intelligent computing cluster to be networked and the number of GPUs connected to each switch; Identify the current network port type of the GPU, and determine the number of switches in each layer of the two-layer topology based on the current network port type of the GPU, the total number of GPUs, and the number of GPUs connected to each switch; Based on the current network port type of the GPU, the interconnection relationship between each GPU and multiple switches is determined, and the GPUs and switches in the intelligent computing cluster to be networked are interconnected according to the interconnection relationship between each GPU and multiple switches, so as to obtain the network topology representation result of the intelligent computing cluster.

2. The intelligent computing cluster networking topology method according to claim 1, characterized in that, The GPU's current network interface type is either a single-port network card or a dual-port network card.

3. The intelligent computing cluster networking topology method according to claim 2, characterized in that, The current network port type of the GPU is the single-port network card type. The interconnection relationship between each GPU and multiple switches is determined based on the current network port type of the GPU. The GPUs and switches in the intelligent computing cluster to be networked are interconnected according to the interconnection relationship between each GPU and multiple switches, resulting in a network topology representation of the intelligent computing cluster, including: Based on the total number of GPUs, the network interface cards (NICs) of each GPU are numbered, and based on the parity classification rule, all GPUs are divided into odd-numbered NICs and even-numbered NICs according to the numbering results. Determine the number of blocks in each topology layer, and determine the number of switches and network interface card ports in each block based on the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each topology layer. When the network interface card (NIC) of the GPU is an even-numbered NIC, a topology link is established for the number of switches and NIC ports in each block based on a preset first interconnection relationship; when the network interface card of the GPU is an odd-numbered NIC, a topology link is established for the number of switches and NIC ports in each block based on a preset second interconnection relationship, thereby obtaining the network topology representation result of the intelligent computing cluster.

4. The intelligent computing cluster networking topology method according to claim 3, characterized in that, The preset first interconnection relationship is: The p-th network interface port is connected to the [(p-1)mod[n / (2]]-th network interface port within the same block. k b1)]]+1 switches, 1≤p≤n / (2 b1), where mod is the modulo operation, k is the number of GPUs connected to each switch, b1 is the number of Blocks in the first layer topology, and n is the total number of GPUs in the intelligent computing cluster to be networked; The preset second interconnection relationship is as follows: The qth network interface port is connected to the [(q-1) mod [n / (2]]th network interface port in the same block. k b2)]]+1 switches, 1≤q≤n / (2 b2), where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of Blocks in the second-layer topology.

5. The intelligent computing cluster networking topology method according to claim 2, characterized in that, The current network port type of the GPU is the dual-port network card type. The interconnection relationship between each GPU and multiple switches is determined based on the current network port type of the GPU. The GPUs and switches in the intelligent computing cluster to be networked are interconnected according to the interconnection relationship between each GPU and multiple switches, resulting in a network topology representation of the intelligent computing cluster, including: Determine the number of blocks in each topology layer, and determine the number of switches and network interface card ports in each block based on the total number of GPUs, the number of GPUs connected to each switch, and the number of blocks in each topology layer. When the network interface card (NIC) port of the GPU is the first port, a topological link is established for the number of switches and NIC ports in each block of the first-layer topology based on a preset third interconnection relationship; when the network interface card (NIC) port of the GPU is the second port, a topological link is established for the number of switches and NIC ports in each block of the second-layer topology based on the preset fourth interconnection relationship, thus obtaining the intelligent computing cluster network topology representation result.

6. The intelligent computing cluster networking topology method according to claim 5, characterized in that, The preset third interconnection relationship is as follows: The x-th network interface port is connected to the [(x-1)mod[n / (k]th] node in the same block. b1)]]+1 switches, 1≤x≤n / b1, where mod is the modulo operation, k is the number of GPUs connected to each switch, b1 is the number of Blocks in the first layer topology, and n is the total number of GPUs in the intelligent computing cluster to be networked; The preset fourth interconnection relationship is as follows: The y-th network interface port is connected to the [(y-1)mod[n / (k]th] node in the same block. b2)]]+1 switches, 1≤y≤n / b2, where mod is the modulo operation, k is the number of GPUs connected to each switch, and b2 is the number of blocks in the second-layer topology.

7. The intelligent computing cluster networking topology method according to claim 1, characterized in that, The two switches in the dual-layer topology are connected in a completely bipartite graph.

8. A smart computing cluster networking topology device, characterized in that, The device includes: The determination module is used to determine the total number of GPUs in the intelligent computing cluster to be networked and the number of GPUs connected to each switch; The computing module is used to identify the current network port type of the GPU, and determine the number of switches in each layer of the two-layer topology based on the current network port type of the GPU, the total number of GPUs, and the number of GPUs connected to each switch. The generation module determines the interconnection relationship between each GPU and multiple switches based on the current network port type of the GPU, and interconnects the GPUs and switches in the intelligent computing cluster to be networked according to the interconnection relationship between each GPU and multiple switches, thereby obtaining the network topology representation result of the intelligent computing cluster.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the intelligent computing cluster networking topology method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the intelligent computing cluster networking topology method as described in any one of claims 1-7.

Images