A GPU interconnection system based on PCIe switch

Abstract

The present application relates to the technical field of GPU cluster design, and in particular to a GPU interconnection system based on a PCIe switch, the system comprising: M PCIe switches, N GPU chip sets, wherein a single GPU chip set comprises M GPU chips, the M GPU chips contained in the single GPU chip set are interconnected through a preset connection relationship, the s-th GPU chip respectively contained in the N GPU chip sets are connected to the s-th PCIe switch, it can be known that the GPU chip sets can be interconnected through a single PCIe switch, multiple PCIe switches can be used to expand the bandwidth of the interconnection between the GPU chip sets without establishing connections between the PCIe switches, and the data transmission efficiency of the GPU cluster can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of GPU cluster design, and in particular to a GPU interconnection system based on a PCIe switch. Background Art

[0002] With the rapid development of fields such as artificial intelligence and machine learning, the application scenarios of GPU clusters are also expanding. In these fields, GPU clusters are widely used in tasks such as image recognition, speech recognition, and natural language processing. Taking natural language processing tasks as an example, large language models have hundreds of billions or even trillions of parameters. Therefore, for the training and reasoning of large language models, the use of GPU clusters for distributed processing can complete the task in a reasonable time. Therefore, how to design an efficient GPU cluster architecture based on application requirements has become a problem that needs to be solved.

[0003] In the prior art, one method is to interconnect multiple cards in a single server through a PCIe switch, and then interconnect the servers through the PCIe switches corresponding to the multiple servers, for example, through the UPI bus between the CPUs connected to each PCIe switch or through the PCIe interface between the PCIe switches. On this basis, there is a method that proposes to add other connection methods such as a bridge board when interconnecting multiple cards in a single server through a PCIe switch, so as to improve the transmission bandwidth of the interconnection of multiple cards in a single server;

[0004] However, since the distributed processing of GPU clusters requires the aggregation of the processing results of each GPU chip, the above-mentioned method of adding other connection methods when interconnecting multiple cards within a single server cannot improve the transmission efficiency of interconnection between servers. When interconnecting servers, if the PCIe interface between PCIe switches is used for data transmission, the data transmission efficiency will be limited by the bandwidth of the PCIe interface between PCIe switches. If the UPI bus between CPUs is used for data transmission, the data transmission efficiency will be limited by the bandwidth of the PCIe interface between the PCIe switch and the CPU.

[0005] Therefore, how to design the GPU cluster architecture to improve the data transmission efficiency of the GPU cluster has become an urgent problem to be solved. Summary of the invention

[0006] In view of the above technical problems, the technical solution adopted by the present invention is:

[0007] A GPU interconnection system based on a PCIe switch, the system comprising: M PCIe switches {a1, a2, ..., a m , …, a M}, a set of N GPU chips {B1, B2, ..., B n , …, B N}, where a m is the mth PCIe switch, where m is an integer in the range [1, M], and B n is the nth GPU chip set, where n is an integer in the range [1, N];

[0008] GPU Chipset B n Including M GPU chips {b n 1, b n 2, …, b n m , …, b n M}, where b n m GPU chip set B n The mth GPU chip in the GPU chip set B n The M GPU chips included are interconnected through a preset connection relationship;

[0009] The sth GPU chip b contained in each of the N GPU chip sets 1 s 、b 2 s , …, b n s , …, b N s They are all connected to the s-th PCIe switch, s∈P, P is the set of all integers in the range [1, M].

[0010] Compared with the prior art, the present invention has obvious beneficial effects. By means of the above technical solution, a GPU interconnection system based on a PCIe switch provided by the present invention can achieve considerable technical advancement and practicality, and has wide industrial utilization value, and has at least the following beneficial effects:

[0011] The present invention provides a GPU interconnection system based on a PCIe switch, the system comprising: M PCIe switches {a1, a2, ..., a m , …, a M}, a set of N GPU chips {B1, B2, ..., B n , …, B N}, where a m is the mth PCIe switch, where m is an integer in the range [1, M], and B n is the nth GPU chip set, n is an integer in the range [1, N], GPU chip set B n Including M GPU chips {bn 1, b n 2, …, b n m , …, b n M}, where b n m GPU chip set B n The mth GPU chip in the GPU chip set B n The M GPU chips included are interconnected through a preset connection relationship, and the sth GPU chip b included in the N GPU chip set is 1 s 、b 2 s , …, b n s , …, b N s They are all connected to the s-th PCIe switch, s∈P, P is the set of all integers in the range [1, M].

[0012] It can be seen that the various GPU chips in the GPU chip set are interconnected using a preset connection relationship, and the GPU chip sets can be interconnected through a single PCIe switch. Multiple PCIe switches can be used to expand the bandwidth of the interconnection between GPU chip sets without the need to establish connections between PCIe switches. In the prior art, the interconnection between GPU chip sets needs to be achieved through connections between PCIe switches and it is difficult to flexibly expand the bandwidth of the interconnection between GPU chip sets. Therefore, the above-mentioned GPU cluster architecture can effectively improve the data transmission efficiency of the GPU cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 A schematic diagram of the architecture of a GPU cluster design in a GPU interconnection system based on a PCIe switch provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0016] This embodiment provides a GPU interconnection system based on a PCIe switch, the system comprising: M PCIe switches {a1, a2, ..., a m , …, a M}, a set of N GPU chips {B1, B2, ..., B n , …, B N}, where a m is the mth PCIe switch, where m is an integer in the range [1, M], and B n is the nth GPU chip set, where n is an integer in the range [1, N];

[0017] GPU Chipset B n Including M GPU chips {b n 1, b n 2, …, b n m , …, b n M}, where b n m GPU chip set B n The mth GPU chip in the GPU chip set B n The M GPU chips included are interconnected through a preset connection relationship;

[0018] The sth GPU chip b contained in each of the N GPU chip sets 1 s 、b 2 s , …, b n s , …, b N s They are all connected to the s-th PCIe switch, s∈P, P is the set of all integers in the range [1, M].

[0019] Among them, the PCIe switch may refer to a customizable, multi-port embedded switch of PCIe, which may support the connection of an upstream port and several downstream ports, and the GPU chip set may refer to a server formed by M GPU chips. The M GPU chips contained in a single GPU chip set may be interconnected through a preset connection relationship, and the preset connection relationship may be realized through a high-speed interconnection interface, an optical communication port, a PCIe port, etc., which are not limited here. In this embodiment, the preset connection relationship is described by taking the high-speed interconnection interface as an example. The preset connection relationship may be realized through a fixed connection topology between GPU chips. At this time, the GPU chips are interconnected through the connection between the high-speed interconnection interfaces, for example, a ring topology structure is adopted, or it may be realized through an external high-speed interconnection interface switch. At this time, the GPU chips are respectively connected to the high-speed interconnection interface switch through the high-speed interconnection interface to realize interconnection.

[0020] See also Figure 1 , is an architectural diagram of a GPU cluster design in a GPU interconnection system based on a PCIe switch provided in an embodiment of the present invention. The architectural diagram takes four GPU chip sets, each GPU chip set including four GPU chips, and the GPU chip sets are interconnected through four PCIe switches as an example.

[0021] Specifically, this embodiment assumes that the number of GPU chips included in each GPU chip set is the same, and the M GPU chips included in a single GPU chip set are equivalent to each other. The implementer can adjust the identification of the GPU chip in the GPU chip set to which it belongs according to actual conditions.

[0022] Implementers can determine the number of s values ​​and the specific value of s according to actual needs. When the number of s values ​​is 1, the interconnection relationship between various GPU chip sets can be achieved through a single PCIe switch. The larger the number of s values, the greater the interconnection bandwidth between various GPU chip sets. The specific value of s is used to determine the PCIe switch used for interconnection and the GPU chip connected to it.

[0023] In a specific embodiment, the system further includes a CPU;

[0024] The M PCIe switches are all connected to the CPU, and the CPU is used to send n m Send the corresponding sub-data c to be processed n m and processing instruction f n m .

[0025] Among them, the CPU can use the PCIe interface to send data to the GPU chip b nm Directly send the corresponding sub-data c to be processed n m and processing instruction f n m , or it can be connected to the GPU chip b through the PCIe interface n m Send processing instruction f n m and memory address, so that the GPU chip b n m Access the memory address to obtain the sub-data c to be processed n m .

[0026] In a specific implementation, the GPU chip b n m According to the processing instruction f n m C n m After processing, we get b n m The corresponding processed sub-data d n m .

[0027] In the scenario where GPU clusters are used to process large model tasks, the sub-data c n m It can refer to some parameters in large model reasoning or training tasks, and the processed sub-data d n m It can refer to the sub-data to be processed c n m The result of forward calculation in inference tasks or reverse gradient calculation in training tasks.

[0028] In a specific implementation, the GPU chip set B n The M GPU chips included are connected to each other through the preset connection relationship. n 1,d n 2, …, d n m , …, d n M The full aggregation operation obtains the processed data e n , the GPU chip set B n The M GPU chips included all contain the processed data e n .

[0029] Among them, d n 1,d n 2, …, d nm , …, d n M The full aggregation operation can refer to the addition of d n 1,d n 2, …, d n m , …, d n M In GPU chip set B n Each GPU chip contains the processed data n .

[0030] In another embodiment, several GPU chips can be selected from each GPU chip set as root chips. In this case, the GPU chip set B n The M GPU chips included are connected by preset relationships to perform d n 1,d n 2, …, d n m , …, d n M Aggregation operation to obtain processed data e n , GPU chip set B n The included root chips all contain processed data n , d n 1,d n 2, …, d n m , …, d n M The aggregation operation can be referred to as n 1,d n 2, …, d n m , …, d n M In GPU chip set B n Each root chip contained in the cluster is aggregated into processed data e n Accordingly, if you need to interconnect GPU chip sets, you need to use a PCIe switch connected to the root chip for interconnection.

[0031] In a specific implementation, the mth PCIe switch a m N GPU chips connected 1 m 、b 2 m , …, b n m , …, b N m , through a m Processed data e1, e2, ..., en ,…,e N A full gather operation is performed to obtain target data, where the target data is used to be sent to the CPU.

[0032] Among them, e1, e2, …, e n ,…,e N The full aggregation operation can refer to e1, e2, ..., e n ,…,e N Aggregated as target data, each PCIe switch used for interconnecting GPU chip sets contains the target data, that is, the acquisition of the target data does not require communication between PCIe switches or communication between CPUs.

[0033] In a specific implementation, M is set to 4.

[0034] In a specific implementation, N is set to 4.

[0035] In a specific implementation manner, the CPU includes a first CPU and a second CPU, a1 and a2 are both connected to the first CPU, and a3 and a4 are both connected to the second CPU.

[0036] Among them, the GPU cluster architecture can be controlled by multiple CPUs.

[0037] In this embodiment, the GPU chips in the GPU chip set are interconnected using a preset connection relationship. The GPU chip sets can be interconnected through a single PCIe switch. Multiple PCIe switches can be used to expand the bandwidth of the interconnection between GPU chip sets without establishing connections between PCIe switches. In the prior art, the interconnection between GPU chip sets needs to be achieved through connections between PCIe switches and it is difficult to flexibly expand the bandwidth of the interconnection between GPU chip sets. Therefore, the above-mentioned GPU cluster architecture can effectively improve the data transmission efficiency of the GPU cluster.

[0038] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A GPU interconnection system based on a PCIe switch, characterized in that: The system comprises: M PCIe switches {a1, a2, ..., a m , …, a M }, a set of N GPU chips {B1, B2, ..., B n , …, B N }, where a m is the mth PCIe switch, where m is an integer in the range [1, M], and B n is the nth GPU chip set, where n is an integer in the range [1, N]; GPU Chipset B n Including M GPU chips {b n 1, b n 2, …, b n m , …, b n M }, where b n m GPU chip set B n The mth GPU chip in the GPU chip set B n The M GPU chips included are interconnected through a preset connection relationship; The sth GPU chip b contained in each of the N GPU chip sets 1 s , b 2 s , …, b n s , …, b N s They are all connected to the s-th PCIe switch, s∈P, P is the set of all integers in the range [1, M].

2. The PCIe switch-based GPU interconnection system according to claim 1, characterized in that: The system also includes a CPU; The M PCIe switches are all connected to the CPU, and the CPU is used to send n m Send the corresponding sub-data c to be processed n m and processing instruction f n m .

3. The GPU interconnection system based on PCIe switch according to claim 2, characterized in that: GPU chip b n m According to the processing instruction f n m C n m After processing, we get b n m The corresponding processed sub-data d n m。 4. The PCIe switch-based GPU interconnection system according to claim 3, characterized in that: GPU Chipset B n The M GPU chips included are connected to each other through the preset connection relationship. n 1,d n 2, …, d n m , …, d n M The full aggregation operation obtains the processed data e n , the GPU chip set B n The M GPU chips included all contain the processed data e n .

5. The PCIe switch-based GPU interconnection system according to claim 4, characterized in that: With the mth PCIe switch a m N GPU chips connected 1 m , b 2 m , …, b n m , …, b N m , through a m Processed data e1, e2, ..., e n ,…,e N A full gather operation is performed to obtain target data, where the target data is used to be sent to the CPU.

6. The PCIe switch-based GPU interconnection system according to claim 1, characterized in that: M is set to 4.

7. The PCIe switch-based GPU interconnection system according to claim 4, characterized in that: N is set to 4.

8. The PCIe switch-based GPU interconnection system according to claim 2, characterized in that: The CPU includes a first CPU and a second CPU, a1 and a2 are both connected to the first CPU, and a3 and a4 are both connected to the second CPU.

Images