GPU cross-server interconnection system

Abstract

The invention relates to the technical field of computers, in particular to a GPU cross-server interconnection system which comprises M servers and N switches, the mth server am comprises Q GPU chips, the Q GPU chips contained in any server are fixedly connected, the nth switch bn is connected with the s (n) servers, the s (n) servers are connected with the M servers, and the N switches are connected with the M servers. The s (n) servers having the connection relation with the bn form a server group corresponding to the preset topological relation. According to the system, when the server or the GPU chip breaks down, only the fault server needs to be processed, the maintenance cost is reduced, the interconnection relation of the GPU chips among multiple servers can be configured through the switch, high-bandwidth interconnection server groups with different capacities are formed, the service requirements are flexibly adapted, the flexibility of a GPU interconnection architecture is improved, and the system is suitable for large-scale popularization and application. While the high interconnection bandwidth is maintained, the switch cost and interconnection lines are saved, and the complexity of switch routing and design is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a GPU cross-server interconnection system. Background Art

[0002] In application scenarios such as large model training, high-bandwidth GPU clusters are usually required to provide support. Common servers based on PCIE interfaces usually use eight GPUs for interconnection. In order to further improve the bandwidth of the GPU cluster, the prior art proposes the use of a ring topology for high-speed interconnection between servers, thereby realizing a scalable high-bandwidth GPU interconnection architecture. In the ring topology, in addition to the connection relationship between the GPU chips contained in the server, there is also a connection relationship between the corresponding GPU chips between each server.

[0003] However, in the existing high-bandwidth ring GPU interconnection architecture, the connection relationship between GPU chips within the server and between servers is implemented using fixed connections. In the event of a GPU chip failure or a server failure, in addition to replacing the GPU chip or server, the chip connection needs to be redeployed, resulting in high fault maintenance costs and long business recovery time. One way to speed up business recovery is to replace faulty machines in units of server groups, but the cost of cluster redundant servers in this way is high. Moreover, although the existing GPU interconnection architecture is logically scalable, when adjusting the GPU interconnection architecture, it is still necessary to redeploy the physical devices of the chip connection, resulting in a huge amount of engineering work to adjust the GPU interconnection architecture of clusters of thousands or even tens of thousands of cards in actual application scenarios.

[0004] Therefore, how to improve the flexibility of the GPU interconnection architecture and reduce the maintenance cost of the GPU interconnection architecture while ensuring its high scalability has become an urgent problem to be solved. Summary of the invention

[0005] In view of the above technical problems, the technical solution adopted by the present invention is:

[0006] A GPU cross-server interconnection system, the system comprising: M servers {a1, a2, ..., a m , …, a M}, N switches {b1, b2, ..., b n , …, b N}, where a m is the mth server, where m is an integer in the range [1, M], and b n is the nth switch, where n is an integer in the range [1, N].

[0007] a m Including Q GPU chips {c m 1, c m 2, …, c m q , …, c m Q}, where c m q is the qth GPU chip in the mth server, q is an integer in the range of [1, Q], and there is a fixed connection relationship between the Q GPU chips contained in any server.

[0008] The nth switch b n There is a connection relationship with s(n) servers, among which, with b n The s(n) servers that are connected form a server group. For any server in the server group, at least one GPU chip and b n There is a connection relationship, there is a corresponding relationship between s(n) and n, s(n) is an integer in the range of [1, M], and s(n) is determined according to a preset topological relationship.

[0009] Compared with the prior art, the present invention has obvious beneficial effects. By means of the above technical solution, a GPU cross-server interconnection system provided by the present invention can achieve considerable technical advancement and practicality, and has wide industrial utilization value, and has at least the following beneficial effects:

[0010] The present invention provides a GPU cross-server interconnection system, the system comprising: M servers {a1, a2, ..., a m , …, a M}, N switches {b1, b2, ..., b n , …, b N}, where a m is the mth server, where m is an integer in the range [1, M], and b n is the nth switch, n is an integer in the range [1, N], a m Including Q GPU chips {c m 1, c m 2, …, c m q , …, c m Q}, where c m q is the qth GPU chip in the mth server, q is an integer in the range [1, Q]. There is a fixed connection relationship between the Q GPU chips contained in any server. The nth switch b n There is a connection relationship with s(n) servers, among which, with bn The s(n) servers that are connected form a server group. For any server in the server group, at least one GPU chip and b n There is a connection relationship, there is a corresponding relationship between s(n) and n, s(n) is an integer in the range of [1, M], and s(n) is determined according to a preset topological relationship.

[0011] It can be seen that the GPU chip connection between servers is realized through multiple switches, so that the GPU interconnection architecture can quickly support the operation of adjusting the number of high-bandwidth interconnection expansion servers, ensuring the high scalability of the GPU interconnection architecture; when a server or GPU chip fails, only the failed server needs to be processed without re-adjusting the wiring of other servers, thereby effectively reducing maintenance costs and shortening business recovery time; moreover, the GPU chip interconnection relationship between multiple servers can be configured through the switch to form a high-bandwidth interconnection server group with different capacities, so as to flexibly adapt to business needs in application scenarios and improve the flexibility of the GPU interconnection architecture. In addition, compared with the method of fully connecting the high-bandwidth interconnection of all GPU chips to the switch in the prior art, the present invention can effectively save switch costs and interconnection lines while maintaining the same high interconnection bandwidth, reduce the complexity of switch routing and design, and thus save power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic diagram of the architecture of a GPU cross-server interconnection system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0015] This embodiment provides a GPU cross-server interconnection system, the system comprising: M servers {a1, a2, ..., a m , …, a M}, N switches {b1, b2, ..., b n , …, b N}, where a m is the mth server, where m is an integer in the range [1, M], and b n is the nth switch, where n is an integer in the range [1, N];

[0016] a m Including Q GPU chips {c m 1, c m 2, …, c m q , …, c m Q}, where c m q is the qth GPU chip in the mth server, q is an integer in the range of [1, Q], and there is a fixed connection relationship between the Q GPU chips contained in any server;

[0017] The nth switch b n There is a connection relationship with s(n) servers, among which, with b n The s(n) servers that are connected form a server group. For any server in the server group, at least one GPU chip and b n There is a connection relationship, there is a corresponding relationship between s(n) and n, s(n) is an integer in the range of [1, M], and s(n) is determined according to a preset topological relationship.

[0018] A single server includes Q GPU chips, and there is a fixed connection relationship between the Q GPU chips belonging to the same server. In this embodiment, the Q GPU chips belonging to the same server are interconnected through fixed lines.

[0019] Specifically, s(n) can be understood as n The capacity of the server group formed by all servers in a connection relationship can be determined based on a preset topology relationship. The preset topology relationship can be a 16-card interconnection topology relationship, a 32-card interconnection topology relationship, etc. The implementer can determine the preset topology relationship based on actual business needs.

[0020] For example, assume that the system includes 8 servers and 8 switches, each server includes 8 GPU chips, and for one switch, when only one GPU chip in a single server is connected to the switch, the switch is connected to 8 GPU chips to form a server group including 8 servers, and each switch realizes the interconnection between the 8 servers. It should be noted that when the switch is connected to the GPU chip in the server, the identification information of the GPU chip in the server can be determined according to the actual situation, and is not limited here;

[0021] For a switch, when two GPU chips in a single server are connected to the switch, a single switch is connected to eight GPU chips to form a server group containing four servers, and each switch realizes the interconnection between four servers.

[0022] The above two methods correspond to different server group capacities. For a server group containing 8 servers, more servers can be interconnected. For a server group containing 4 servers, the interconnection between GPU chips in a single server can obtain bandwidth enhancement through a switch.

[0023] When any GPU chip fails, the GPU chip can be replaced. At this time, it is only necessary to redeploy the fixed connection relationship between each GPU chip in the server to which the GPU chip belongs, as well as the connection relationship between the replaced GPU chip and the switch. Compared with the existing GPU interconnection architecture, when a GPU chip fails, it is necessary to redeploy the connection relationship between the GPU chips in all servers connected to the GPU chip. Similarly, when any server fails, it is only necessary to redeploy the connection relationship between each GPU chip in the replaced server and the switch, thereby greatly reducing the maintenance cost and time of the GPU interconnection architecture, thereby reducing the service recovery time of the GPU interconnection architecture.

[0024] See also Figure 1 , is an architectural diagram of a GPU cross-server interconnection system provided in an embodiment of the present invention. In the architectural diagram, two servers are taken as an example, each server contains four GPU chips, and the two servers are connected through four switches.

[0025] In a specific implementation, any GPU chip includes K interconnection ports, where K is an integer greater than zero;

[0026] For any switch b n There is a GPU chip in a connection relationship, wherein the GPU chip is connected to b through an interconnection port included in the GPU chip n connect.

[0027] In order to further improve the bandwidth of the GPU cluster, a single GPU chip can provide multiple interconnection ports to interconnect with GPU chips of other servers. For a GPU chip including K interconnection ports, the GPU chip can be connected to K switches.

[0028] For example, assuming that the system includes 8 servers and 16 switches, each server contains 8 GPU chips, and each GPU chip contains two interconnection ports. Then, one of the interconnection ports can be connected to the 8 switches to interconnect the 8 servers, and the other interconnection port can be connected to the other 8 switches to interconnect the 8 servers, thereby improving the bandwidth of the interconnection between the 8 servers.

[0029] In a specific implementation, each switch includes P interconnection ports, and P satisfies the condition: P≥M.

[0030] The interconnection ports included in the switch can represent the maximum number of server interconnections supported by the switch. Accordingly, the number of interconnection ports included in the switch should be greater than or equal to the number of servers.

[0031] Specifically, the number of servers connected to the switch is determined according to the actual topology. For example, for a switch, when there is only one GPU chip connected to the switch in a server in the topology, the number of servers connected to the switch may be P. When there are Z GPU chips connected to the switch in a server in the topology, the number of servers connected to the switch may be P / Z. Under this topology, the GPU chips in a single server can also be interconnected on the switch to obtain bandwidth enhancement, thereby further increasing the flexibility of the topology.

[0032] In a specific implementation, P is set to 32.

[0033] In this embodiment, P is set to 32, that is, the GPU interconnection architecture provided by this embodiment supports a maximum of 32 server interconnections.

[0034] In a specific implementation, when any GPU chip is connected to its corresponding switch, if the distance of the connection between the GPU chip and its corresponding switch is greater than a preset distance threshold, an optical cable is used for the connection between the GPU chip and its corresponding switch; if the distance of the connection between the GPU chip and its corresponding switch is greater than a preset distance threshold, an electrical cable is used for the connection between the GPU chip and its corresponding switch.

[0035] Among them, in order to ensure the efficiency of data transmission of the GPU cluster, if the distance between the connection between the GPU chip and its corresponding switch is greater than the preset distance threshold, an optical cable is used for optical communication, otherwise an electric cable is used. Compared with the GPU interconnection architecture that completely uses optical cables, the cost is effectively reduced. In addition, in this embodiment, the GPU chip only uses a switch when interconnected across servers, and the GPU interconnection within the server still uses a fixed connection method. Compared with the method of fully connecting the high-bandwidth interconnection of all GPU chips to the switch in the prior art, this embodiment can maintain the same high interconnection bandwidth while effectively saving switch costs and interconnection lines, reducing the complexity of switch routing and design, and thus saving power consumption.

[0036] In a specific implementation, the M servers include R interconnected servers and T standby servers, the R interconnected servers form an interconnected relationship between the R interconnected servers through the N switches, and R and T satisfy the condition: R+T=M.

[0037] In order to further improve the service recovery speed of the GPU interconnection architecture, several backup servers are set up among the M servers. Only when an interconnection server fails, the backup server is configured to directly join the GPU interconnection architecture to continue to support service processing.

[0038] In this embodiment, the servers are divided into interconnected servers and standby servers. When an interconnected server or a GPU chip therein fails, only the failed server needs to be taken offline, and other standby servers connected to the switch are selected through the switch to form a new server group, without re-adjusting the wiring of other servers, thereby effectively reducing maintenance costs and shortening service recovery time.

[0039] In a specific implementation manner, the switch is further used to configure the interconnection relationship of the M servers by configuring the data transmission relationship between each GPU chip connected to the switch.

[0040] The switch can control whether the GPU chips are connected by configuring the routing method between the ports it contains. Implementers can flexibly build the interconnection relationship between GPU chips according to the preset topology relationship based on actual business needs. When business needs are adjusted, the interconnection topology with a single server as the smallest unit can be adjusted only through switch configuration, which effectively improves the flexibility of the GPU interconnection architecture.

[0041] In a specific implementation manner, the type corresponding to the switch at least includes a PCIE switch type.

[0042] Among them, the types corresponding to the switch may also include CXL switch type, high-speed interconnect switch type, OCS optical switch type, etc. It should be noted that the type corresponding to the switch should be adapted to the communication protocol corresponding to the interconnection interface it contains.

[0043] In this embodiment, the GPU chip connection between servers is implemented through multiple switches, so that the GPU interconnection architecture can quickly support the operation of adjusting the number of high-bandwidth interconnection expansion servers, ensuring the high scalability of the GPU interconnection architecture; when a server or GPU chip fails, only the failed server needs to be processed without re-adjusting the wiring of other servers, thereby effectively reducing maintenance costs and shortening business recovery time; moreover, the GPU chip interconnection relationship between multiple servers can be configured through the switch to form a high-bandwidth interconnection server group with different capacities, so as to flexibly adapt to business needs in application scenarios and improve the flexibility of the GPU interconnection architecture. In addition, compared with the method of fully connecting the high-bandwidth interconnection of all GPU chips to the switch in the prior art, the present invention can effectively save switch costs and interconnection lines while maintaining the same high interconnection bandwidth, reduce the complexity of switch routing and design, and thus save power consumption.

[0044] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A GPU cross-server interconnection system, characterized in that: The system includes: M servers {a1, a2, ..., a m , …, a M }, N switches {b1, b2, ..., b n , …, b N }, where a m is the mth server, where m is an integer in the range [1, M], and b n is the nth switch, where n is an integer in the range [1, N]; a m Including Q GPU chips {c m 1, c m 2, …, c m q , …, c m Q }, where c m q is the qth GPU chip in the mth server, q is an integer in the range of [1, Q], and there is a fixed connection relationship between the Q GPU chips contained in any server; The nth switch b n There is a connection relationship with s(n) servers, among which, with b n The s(n) servers that are connected form a server group. For any server in the server group, at least one GPU chip and b n There is a connection relationship, there is a corresponding relationship between s(n) and n, s(n) is an integer in the range of [1, M], and s(n) is determined according to a preset topological relationship.

2. The GPU cross-server interconnection system according to claim 1, characterized in that: Any GPU chip contains K interconnection ports, where K is an integer greater than zero; For any switch b n There is a GPU chip in a connection relationship, wherein the GPU chip is connected to b through an interconnection port included in the GPU chip n connect.

3. The GPU cross-server interconnection system according to claim 2, characterized in that: Each switch contains P interconnected ports, and P satisfies the condition: P ≥ M.

4. The GPU cross-server interconnection system according to claim 3, characterized in that: P is set to 32.

5. The GPU cross-server interconnection system according to claim 1, characterized in that: When any GPU chip is connected to its corresponding switch, if the distance between the GPU chip and the corresponding switch is greater than a preset distance threshold, the connection between the GPU chip and the corresponding switch adopts an optical cable; If the distance of the connection between the GPU chip and its corresponding switch is greater than a preset distance threshold, the connection between the GPU chip and its corresponding switch adopts a cable.

6. The GPU cross-server interconnection system according to claim 1, characterized in that: The M servers include R interconnected servers and T standby servers. The R interconnected servers form a server group corresponding to the R interconnected servers through the N switches. R and T satisfy the condition: R+T=M.

7. The GPU cross-server interconnection system according to claim 1, characterized in that: The switch is also used to configure the interconnection relationship of the M servers by configuring the data transmission relationship between each GPU chip connected to the switch.

8. The GPU cross-server interconnection system according to claim 1, characterized in that: The types corresponding to the switch include at least a PCIE switch type.

Images