Computing interconnection system and node server

Abstract

The application discloses a kind of computing interconnection system and node server, it is related to computer technical field, the computing interconnection system includes a computing node and m switch, computing node includes n processor;At least including first switching device and second switching device in switch, first switching device is connected with k processor in n processor, and second switching device is connected with other k processor except k processor in n processor;Or, computing node further includes multiple first rate conversion device, the input of first rate conversion device is connected with n processor respectively, and the output of first rate conversion device is connected with switch, and first rate conversion device is used to convert the rate of signal output by processor from first rate to second rate, and second rate is integer times of first rate.The application solves the technical problem that super-node server cannot realize node expansion due to the limitation of switch bandwidth in the related art.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a computing interconnection system and node server. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, the training and inference needs of large-scale deep learning models place high demands on computing resources. AI supernode servers, through a fully interconnected architecture, enable efficient communication between multiple graphics processing units (GPUs), which can meet the complex computing needs of distributed training, such as data parallelism and model parallelism.

[0003] In practical applications, the Peripheral Component Interconnect Express (PCIe) interconnect protocol has become a technical solution for improving computing efficiency due to its low latency and low protocol overhead. However, among related technical solutions, the PCIe interconnect solution is limited by the switch bandwidth and cannot achieve external node expansion. Summary of the Invention

[0004] This application provides a computing interconnection system and node server that can solve the technical problem in related technologies where the limitation of switch bandwidth prevents supernode servers from achieving external node expansion.

[0005] In a first aspect, this application provides a computing interconnect system, which includes a computing nodes and m switches, and the computing nodes include n processors; a, m, and n are positive integers.

[0006] A switch includes at least a first switching unit and a second switching unit. The first switching unit is connected to k processors out of n processors, and the second switching unit is connected to the other k processors out of the n processors. The k processors connected to the first switching unit in different switches are not identical, and the k processors connected to the second switching unit in different switches are also not identical; k is a positive integer, and... .

[0007] Alternatively, the computing node may also include multiple first rate conversion devices, the input terminals of which are connected to n processors respectively, and the output terminals of which are connected to a switch. The first rate conversion devices are used to convert the rate of the signals output by the processors from a first rate to a second rate, wherein the second rate is an integer multiple of the first rate.

[0008] Secondly, this application also provides a node server that includes the computing interconnect system as provided in the first aspect.

[0009] The computing interconnect system and node server provided in this application, by providing a new PCIe interconnect architecture design, can break through the bandwidth limitation of PCIe switches, realize the increase in the number of GPUs in the supernode server, and support the expansion interconnection inside and outside the node. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the architecture of a PCIE-based supernode server provided in the embodiments of this application;

[0012] Figure 2 This is a schematic diagram of the structure of a computing interconnection system provided in the embodiments of this application. Figure 1 ;

[0013] Figure 3 This is a schematic diagram of the structure of a computing interconnection system provided in the embodiments of this application. Figure 2 ;

[0014] Figure 4 This is a bandwidth allocation table for interconnection between GPUs and the switching device in the embodiments of this application;

[0015] Figure 5 This is a schematic diagram of the interconnection between chassis provided in an embodiment of this application;

[0016] Figure 6 This is a schematic diagram of the structure of a computing interconnection system provided in the embodiments of this application. Figure 3 ;

[0017] Figure 7 This is a schematic diagram of the structure of a computing interconnection system provided in the embodiments of this application. Figure 4 . Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0019] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] In this embodiment of the application, "multiple" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0021] In the highly integrated computing environment of AI supernode servers, multiple GPUs, through sophisticated hardware architecture and software collaboration, achieve a fully interconnected topology, enabling them to execute highly efficient parallel computing tasks. This design not only effectively improves data processing speed but also provides powerful computing support for the training and inference of complex AI models.

[0022] When implementing scale-up interconnects, some GPUs primarily employ two different types of protocols. The first is based on Ethernet. Ethernet, as a widely deployed and mature network communication technology, not only boasts high bandwidth transmission capabilities, easily handling the real-time transmission needs of massive amounts of data, but also, due to its multi-machine interconnect capabilities, has become an ideal choice for connecting multiple GPU nodes to form ultra-large-scale computing clusters. In a fully interconnected Ethernet-based architecture, each GPU node can communicate directly with other nodes via high-speed Ethernet links, achieving efficient data sharing and collaborative task processing.

[0023] The second type is the interconnection scheme based on PCIe. As a mature high-speed serial computer expansion bus standard, PCIe plays an important role in direct communication between GPUs within a single machine due to its low latency and high bandwidth.

[0024] While Ethernet-based solutions currently hold a significant market share in the AI ​​supernode server market due to their mature technological ecosystem, widespread application base, and excellent multi-machine interconnection capabilities, making them the preferred choice for many data centers and AI computing scenarios, the increasing demands of AI models for computational performance, especially the urgent need for low-latency communication, are gradually making PCIe interconnect-based AI supernode servers a new focus of technological innovation.

[0025] During the computation of large AI models, GPUs need to frequently exchange data and perform synchronization operations. Even the slightest latency can become a bottleneck restricting overall computing performance. The low latency of PCIe perfectly meets this requirement, effectively reducing communication latency and ensuring efficient and fast data transmission between GPUs. Furthermore, the overhead of the PCIe protocol is relatively low, meaning that more bandwidth can be used for actual data transmission rather than protocol processing, further improving computing performance.

[0026] Based on the aforementioned advantages of PCIe, developing supernode servers that scale upwards using PCIe interconnects has become a trend. These servers achieve direct interconnection between GPUs via PCIe, creating a high-efficiency, low-latency computing environment that provides powerful computing support for training large AI models.

[0027] In related technologies, a supernode server can contain multiple computing nodes, each equipped with several GPUs. These GPUs are the core units for executing AI computing tasks. To achieve full interconnection between GPUs, the supernode server can also incorporate multiple PCIe switches, forming a non-blocking interconnect architecture. In this architecture, any two GPUs can communicate directly without intermediary nodes, thus reducing communication latency.

[0028] In terms of physical form, this PCIe-based supernode design fully considers the convenience and efficiency of actual deployment. Compute nodes and PCIe switches can be interconnected via cable trays, a design that makes cable layout neater and more organized, facilitating maintenance and management. Furthermore, compute nodes and switches can employ an orthogonal architecture with 90-degree interlocking, which not only saves space but also improves connection stability and reliability.

[0029] For example, refer to Figure 1 , Figure 1 This is a schematic diagram of the architecture of a PCIE-based supernode server provided in the embodiments of this application.

[0030] In some implementations, the supernode server includes multiple compute nodes (e.g., compute node 1 to compute node 8) and multiple connectors (e.g., connector 1 to connector 6), with each compute node integrating multiple GPUs (e.g., GPU1 to GPU4). The GPUs within the compute nodes can be initially interconnected via a PCIe bus to ensure efficient data exchange within the node.

[0031] The aforementioned supernode server also includes multiple PCIe switches (such as switches 1 to 6), each of which includes a switching device. Specifically, the switching device in switch 1 is connected to connector 1 in each compute node, the switching device in switch 2 is connected to connector 2 in each compute node, and so on, and the switching device in switch 6 is connected to connector 6 in each compute node.

[0032] The above connection method enables full interconnection of GPUs across computing nodes, constructing a non-blocking network topology. Each GPU can communicate with other GPUs via single-hop or multi-hop paths.

[0033] However, due to the inherent limitations in bandwidth design of PCIe switches, their technical characteristics directly restrict the scalability and computing density of supernode servers.

[0034] For example, the PCIe protocol uses a lane-based bandwidth allocation model, with each lane (x1) providing unidirectional transmission capability. To ensure computing efficiency, each GPU needs to occupy at least x8 lanes of bandwidth, and a typical PCIe switch supports a maximum of x256 lanes. Based on an x8 port configuration, a single PCIe switch can connect a maximum of 32 GPUs, forming a hard connection limit of 32 cards.

[0035] When all 32 GPUs are communicating at full speed, all channel resources of the PCIe switch will be fully occupied, the total throughput will reach the theoretical peak, and there will be no remaining bandwidth available for external device access.

[0036] like Figure 1 As shown, assuming each GPU requires x8 channels of bandwidth and the maximum bandwidth supported by the PCIe switch is x256, each PCIe switch can only connect a maximum of 32 GPUs. Therefore, the maximum computing capacity of the aforementioned supernode server is only 32 cards. Furthermore, since the GPUs within the supernode server have already fully utilized the bandwidth of the PCIe switch, the switch has no additional bandwidth to extend to other servers outside the supernode server, thus preventing external expansion.

[0037] To address the aforementioned technical issues, this application provides a computing interconnect system and node server. By providing a new PCIe interconnect architecture design, it can overcome the bandwidth limitations of PCIe switches, enabling an increase in the number of GPUs within the supernode server (e.g., from 32 cards to 64 cards), while also supporting extended interconnection inside and outside the node.

[0038] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] In Embodiment 1, this application provides a computing interconnect system, which includes a computing nodes and m switches, and each computing node includes n processors; where a, m, and n are positive integers.

[0040] Each of the above switches includes at least a first switching device and a second switching device. The first switching device is connected to k processors out of the above n processors, and the second switching device is connected to the other k processors out of the above n processors.

[0041] In this context, the k processors connected to the first switching unit in different switches are not exactly the same, and the k processors connected to the second switching unit in different switches are not exactly the same; k is a positive integer, and k≤n / 2.

[0042] Optionally, the processor mentioned above can be a GPU.

[0043] Optionally, the aforementioned switch can be a PCIe switch.

[0044] In some implementations, each of the above computing nodes further includes m connectors; each connector is connected to the above n processors.

[0045] In some implementations, the first switching device in the switch is connected to k of the n processors via its corresponding connector, and the second switching device in the switch is connected to the other k processors among the n processors besides the k processors via its corresponding connector.

[0046] Among them, the i-th switch among the m switches is connected to the i-th connector among the m connectors.

[0047] In some implementations, the above n processors include m processor combinations, and each processor combination includes k processors.

[0048] For example, the above n processors can be arranged and combined to generate m processor combinations, each of which includes k processors; wherein the k processors in different processor combinations are not exactly the same.

[0049] In this configuration, the first switching device in the i-th switch is connected to k processors in the i-th processor combination, and the second switching device in the i-th switch is connected to k processors in the (m-i+1)-th processor combination.

[0050] In some embodiments, the number Q of high-speed serial computer extended bus data transmission channels supported by the first and second switching devices satisfies the following condition: Where p represents the processor's smallest bandwidth unit.

[0051] In some implementations, the interconnect bandwidth between processors with the same label within different computing nodes is ; and / or, the interconnect bandwidth between processors with different labels within the same compute node is ; and / or, the interconnect bandwidth between processors with different labels within different compute nodes is .

[0052] In some implementations, the number of switches m satisfies the following condition: m = C(n, k); where C(n, k) represents the number of combinations of selecting k processors from n different processors.

[0053] For example, when n=4 and k=2, m=C(4,2)=6.

[0054] In some implementations, the total bandwidth of the processor described above is g satisfies the following conditions: .

[0055] For example, see Figure 2 , Figure 2 This is a schematic diagram of the structure of a computing interconnection system provided in the embodiments of this application. Figure 1 .

[0056] exist Figure 2 In the above-mentioned computing interconnect system, there are 8 computing nodes, namely computing node 1 to computing node 8; each computing node includes 4 GPUs, namely GPU1 to GPU4; each GPU has a total of x8×6 PCIe lanes.

[0057] In some implementations, the computing interconnect system described above may include six switches, namely switch 1 to switch 6.

[0058] In some implementations, in order to enable each switch to have additional bandwidth to connect to the outside of the supernode server in addition to achieving full interconnection of each GPU, each switch may include two switching devices, such as a first switching device (SW1) and a second switching device (SW2).

[0059] Understandably, although the aforementioned switch has two switching units, due to its fully interconnected nature, all GPUs must be connected to the same switch using the minimum bandwidth unit for each GPU. In other words, even if the switch has multiple switching units, each switching unit must connect to all GPUs; that is, each switching unit must connect all 256 channels to the GPUs inside the supernode server.

[0060] To enable each switch to have additional bandwidth connecting to the external supernode server, some implementations may set up six connectors per compute node. Each connector may contain x8 signals for each GPU within the compute node, totaling 32 channels and implementing 64 differential pairs (DP).

[0061] In some implementations, the above four GPUs can be arranged and combined to generate six GPU combinations, each of which includes two GPUs; wherein the two GPUs in different GPU combinations are not exactly the same.

[0062] For example, GPU1, GPU2, GPU3, and GPU4 can be combined in the following ways: GPU1 and GPU2, GPU1 and GPU3, GPU1 and GPU4, GPU2 and GPU3, GPU2 and GPU4, and GPU3 and GPU4.

[0063] In some implementations, for the switch (switch 1) corresponding to the first connector in each compute node, its SW1 can connect the x8 signals of GPU1 and GPU2 in each compute node, for a total of 128 channels; its SW2 connects the x8 signals of GPU3 and GPU4 in each compute node, for a total of 128 channels.

[0064] Among them, Figure 2 In the table, N1_GPU1,2 represents GPU1 and GPU2 in compute node 1, N2_GPU1,2 represents GPU1 and GPU2 in compute node 2, ..., N8_GPU1,2 represents GPU1 and GPU2 in compute node 8.

[0065] Similarly, N1_GPU3,4 represents GPU3 and GPU4 in compute node 1, N2_GPU3,4 represents GPU3 and GPU4 in compute node 2, ..., N8_GPU3,4 represents GPU3 and GPU4 in compute node 8.

[0066] Understandably, since both the first and second switching devices have a bandwidth of 256 channels, they each have an additional 128 channels of bandwidth that can be used to connect external computing devices. That is, both the downlink signal connecting to the computing node and the uplink signal connecting to the external supernode have 128 channels, resulting in an uplink / downlink convergence ratio of 1:1.

[0067] Similarly, for the switch (switch 2) corresponding to the second connector in each compute node, its SW1 can connect the x8 signals of GPU1 and GPU3 in each compute node; its SW2 connects the x8 signals of GPU2 and GPU4 in each compute node.

[0068] Among them, Figure 2 In the table, N1_GPU1,3 represents GPU1 and GPU3 in compute node 1, N2_GPU1,3 represents GPU1 and GPU3 in compute node 2, ..., N8_GPU1,3 represents GPU1 and GPU3 in compute node 8.

[0069] Similarly, N1_GPU2,4 represents GPU2 and GPU4 in compute node 1, N2_GPU2,4 represents GPU2 and GPU4 in compute node 2, ..., N8_GPU2,4 represents GPU2 and GPU4 in compute node 8.

[0070] For the switch (switch 3) corresponding to the third connector in each compute node, its SW1 can connect the x8 signals of GPU1 and GPU4 in each compute node; its SW2 connects the x8 signals of GPU2 and GPU3 in each compute node.

[0071] Among them, Figure 2 In the table, N1_GPU1,4 represents GPU1 and GPU4 in compute node 1, N2_GPU1,4 represents GPU1 and GPU4 in compute node 2, ..., N8_GPU1,4 represents GPU1 and GPU4 in compute node 8.

[0072] Similarly, N1_GPU2,3 represents GPU2 and GPU3 in compute node 1, N2_GPU2,3 represents GPU2 and GPU3 in compute node 2, ..., N8_GPU2,3 represents GPU2 and GPU3 in compute node 8.

[0073] For the switch (switch 4) corresponding to the 4th connector in each compute node, its SW1 can connect the x8 signals of GPU2 and GPU3 in each compute node; its SW2 connects the x8 signals of GPU1 and GPU4 in each compute node.

[0074] Among them, Figure 2 In the table, N1_GPU2,3 represents GPU2 and GPU3 in compute node 1, N2_GPU2,3 represents GPU2 and GPU3 in compute node 2, ..., N8_GPU2,3 represents GPU2 and GPU3 in compute node 8.

[0075] Similarly, N1_GPU1,4 represents GPU1 and GPU4 in compute node 1, N2_GPU1,4 represents GPU1 and GPU4 in compute node 2, ..., N8_GPU1,4 represents GPU1 and GPU4 in compute node 8.

[0076] For the switch (switch 5) corresponding to the 5th connector in each compute node, its SW1 can connect the x8 signals of GPU2 and GPU4 in each compute node; its SW2 connects the x8 signals of GPU1 and GPU3 in each compute node.

[0077] Among them, Figure 2 In the table, N1_GPU2,4 represents GPU2 and GPU4 in compute node 1, N2_GPU2,4 represents GPU2 and GPU4 in compute node 2, ..., N8_GPU2,4 represents GPU2 and GPU4 in compute node 8.

[0078] Similarly, N1_GPU1,3 represents GPU1 and GPU3 in compute node 1, N2_GPU1,3 represents GPU1 and GPU3 in compute node 2, ..., N8_GPU1,3 represents GPU1 and GPU3 in compute node 8.

[0079] For the switch (switch 6) corresponding to the 6th connector in each compute node, its SW1 can connect the x8 signals of GPU3 and GPU4 in each compute node; its SW2 connects the x8 signals of GPU1 and GPU2 in each compute node.

[0080] Among them, Figure 2 In the table, N1_GPU3,4 represents GPU3 and GPU4 in compute node 1, N2_GPU3,4 represents GPU3 and GPU4 in compute node 2, ..., N8_GPU3,4 represents GPU3 and GPU4 in compute node 8.

[0081] Similarly, N1_GPU1,2 represents GPU1 and GPU2 in compute node 1, N2_GPU1,2 represents GPU1 and GPU2 in compute node 2, ..., N8_GPU1,2 represents GPU1 and GPU2 in compute node 8.

[0082] The above topology allows for a single-hop interconnect between any GPU in the computing interconnect system and any other GPU.

[0083] The interconnect bandwidth between GPUs with the same label within different compute nodes is 6x8, the interconnect bandwidth between GPUs with different labels within the same compute node is 2x8, and the interconnect bandwidth between GPUs with different labels within different compute nodes is 2x8.

[0084] For example, refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of a computing interconnection system provided in the embodiments of this application. Figure 2 .

[0085] For the interconnect bandwidth between GPUs with the same label, for example, taking GPU1 in each compute node as an example, they can be interconnected x8 through the first switching device of switch 1, x8 through the first switching device of switch 2, x8 through the first switching device of switch 3, x8 through the second switching device of switch 4, x8 through the second switching device of switch 5, and x8 through the second switching device of switch 6.

[0086] For example, since the first switching device in switch 1 connects the x8 signals of GPU1 and GPU2 in all computing nodes, GPU1 and GPU2 in each computing node can be directly interconnected, achieving x8 channel bandwidth through the first switching device of switch 1; in addition, GPU1 and GPU2 in each computing node can also achieve x8 bandwidth interconnection through the second switching device of switch 6, so the interconnection bandwidth between GPUs with different labels is 2x8.

[0087] For example, refer to Figure 4 , Figure 4 This is a bandwidth allocation table for interconnection between GPUs and the switching device in the embodiments of this application.

[0088] In some implementations, it is assumed that there are a computing nodes in total, and each computing node includes n GPUs. Based on the premise that a×n×p (p is the smallest bandwidth unit of the GPU) = the bandwidth of a single switch, in order to ensure that half of the bandwidth of each switch can be routed out of the supernode, the number of switches m satisfies the following condition: m=C(n,k).

[0089] Here, C(n, k) represents the number of combinations of selecting k processors from n distinct processors. In this case, m is the minimum number of switches required to achieve full interconnection of all GPUs.

[0090] For example, suppose each computing node has 4 GPUs: GPU1, GPU2, GPU3, and GPU4. The minimum condition for achieving full interconnection of all GPUs is: GPU1 interconnected with GPU2, GPU1 interconnected with GPU3, GPU1 interconnected with GPU4, GPU2 interconnected with GPU3, GPU2 interconnected with GPU4, and GPU3 interconnected with GPU4. This requires 6 switches. This is the minimum number of switches needed to achieve full interconnection of all GPUs. The maximum number of switches in practice can be an integer multiple of this value, such as 12 or 18, etc., and this embodiment does not impose a limitation.

[0091] In some implementations, when the number of switches is fixed, each switch can have a smaller number of switching devices, such as two switching devices per switch, or multiple switching devices (such as three) per switch, depending on whether the number of pins on the connector can accommodate the increase in connection signals caused by multiple switching devices in a single switch.

[0092] In order to ensure that each switching device has spare bandwidth that can be routed out of the supernode, the number of switching devices in each switch needs to be greater than or equal to 2.

[0093] In some implementations, the total bandwidth of each GPU is set to k × p (where p is the smallest granularity of GPU bandwidth), then k ≥ C(n, 2) / 2. If k < C(n, 2) / 2, it means that the number of interconnect signal groups brought out by each GPU is insufficient, making it impossible to interconnect with other GPUs.

[0094] In Embodiment 2, the computing interconnect system described above may include multiple chassis, and each chassis includes multiple switches.

[0095] For example, the above-mentioned computing interconnection system includes at least a first chassis and a second chassis; the first chassis and the second chassis each include m switches; the i-th switch in the first chassis is connected to the i-th switch in the second chassis.

[0096] In some implementations, the first switching device of the i-th switch in the first chassis is connected to the first switching device of the i-th switch in the second chassis; the second switching device of the i-th switch in the first chassis is connected to the second switching device of the i-th switch in the second chassis.

[0097] For example, refer to Figure 5 , Figure 5 This is a schematic diagram of the interconnection between chassis provided in an embodiment of this application.

[0098] exist Figure 5 In this system, a high-speed cable can be used to connect the switches in the first chassis and the second chassis, thereby achieving interconnection of all GPUs in the first chassis and the second chassis.

[0099] In this configuration, switch 1 in the first chassis is connected to switch 1 in the second chassis, switch 2 in the first chassis is connected to switch 2 in the second chassis, switch 3 in the first chassis is connected to switch 3 in the second chassis, and so on.

[0100] In some implementations, SW1 of switch 1 in the first chassis is connected to SW1 of switch 1 in the second chassis, and SW2 of switch 1 in the first chassis is connected to SW2 of switch 1 in the second chassis; SW1 of switch 2 in the first chassis is connected to SW1 of switch 2 in the second chassis, and SW2 of switch 2 in the first chassis is connected to SW2 of switch 2 in the second chassis; ...; and so on, thereby achieving full interconnection of all GPUs.

[0101] In Embodiment 3, the above-mentioned computing interconnection system includes a computing nodes and m switches. Each computing node includes n processors; a, m, and n are positive integers. The computing nodes also include multiple first rate conversion devices. The input terminals of the first rate conversion devices are respectively connected to the n processors, and the output terminals of the first rate conversion devices are connected to the switches. The first rate conversion devices are used to convert the rate of the signal output by the processor from a first rate to a second rate, where the second rate is an integer multiple of the first rate.

[0102] In some implementations, the computing node further includes multiple connectors; the i-th switch is connected to the output of the i-th first rate conversion device via the i-th connector.

[0103] In some embodiments, the signal output by the processor is a first high-speed serial computer extended bus signal; the first rate conversion device is used to convert the first high-speed serial computer extended bus signal output by the processor into a second high-speed serial computer extended bus signal; the rate of the second high-speed serial computer extended bus signal is greater than the rate of the first high-speed serial computer extended bus signal.

[0104] In some implementations, the first rate conversion device is used to receive n1 channels of the first high-speed serial computer expansion bus signals from n processors, and to output signals to a connector connected thereto. The switch supports n1 high-speed serial computer expansion bus data transmission channels.

[0105] In some embodiments, the switch further includes a second rate conversion device; the second rate conversion device is used to convert the rate of the signal output by the switch from a second rate to a first rate.

[0106] The bandwidth saved by the second rate conversion device can be used for outward expansion to achieve interconnection with other computing devices.

[0107] For example, refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of a computing interconnection system provided in the embodiments of this application. Figure 3 .

[0108] exist Figure 6 In this system, several first-rate conversion devices (Gearboxes) can be added inside the compute node. These Gearboxes, located between the GPU and the connector, function to adjust the speed. For example, a PCIe Gen5 signal output from the GPU can be converted to a PCIe Gen6 signal after processing by a Gearbox.

[0109] Since the speed of PCIe Gen5 is 32GT / s and the speed of PCIe Gen6 is 64GT / s, the speed of PCIe Gen6 is twice that of PCIe Gen5. The output of the first rate conversion device only needs half the number of signals at the input end to achieve lossless transmission.

[0110] like Figure 6 As shown, each first-rate conversion device receives x8 PCIe Gen5 signals from four GPUs at its input, totaling 32 channels. The output of each first-rate conversion device connects to a connector, providing 16 channels of PCIe Gen6 signals. Therefore, each switch connects a total of 128 channels of signals to the compute nodes, occupying half of the switch's bandwidth. This allows the switch to have half its bandwidth remaining for external node expansion. For example, it can achieve a 1:1 bandwidth ratio between internal (scale up) and external (scale out) operations, resulting in a non-blocking and non-converging effect.

[0111] In some embodiments, multiple second-rate conversions can be configured on the switch to convert the switch's PCIE GEN6 signals to PCIE GEN5 signals, thus enabling the output of 256 channels of PCIE GEN6 signals and achieving bandwidth expansion.

[0112] In Embodiment 4, the processor density can be further increased based on Embodiment 3. The aforementioned computing node includes multiple acceleration modules, each containing at least two processors; the i-th connector is connected to the i-th first rate conversion device, and the first rate conversion device is connected to the processors in the multiple acceleration modules.

[0113] Optionally, the above acceleration module can be an Open Accelerator Module (OAM).

[0114] In some implementations, the switches mentioned above include a third switching device and a fourth switching device respectively; the third switching device in the i-th switch is connected to the i-th connector in a / 2 of the a computing nodes, and the fourth switching device in the i-th switch is connected to the i-th connector in the remaining a / 2 computing nodes.

[0115] In some embodiments, the above-mentioned switch further includes a fifth switching device and a sixth switching device; the input terminal of the fifth switching device is connected to the first output terminal of the third switching device and the first output terminal of the fourth switching device, respectively; the input terminal of the sixth switching device is connected to the second output terminal of the third switching device and the second output terminal of the fourth switching device, respectively.

[0116] In some implementations, the bandwidth of the first output terminal of the third switching device and the bandwidth of the second output terminal are each half of the uplink bandwidth of the third switching device; the bandwidth of the first output terminal of the fourth switching device and the bandwidth of the second output terminal are each half of the uplink bandwidth of the fourth switching device.

[0117] The uplink bandwidth of the fifth switching device is the same as its downlink bandwidth, and the uplink bandwidth of the sixth switching device is the same as its downlink bandwidth; the uplink bandwidth of the fifth switching device and the uplink bandwidth of the sixth switching device are externally extended bandwidths used to realize interconnection with other computing devices.

[0118] For example, refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of a computing interconnection system provided in the embodiments of this application. Figure 4 .

[0119] exist Figure 7 In this embodiment, each computing node is equipped with 4 acceleration modules (OAM1~OAM4 respectively), and each acceleration module includes 2 GPUs. Thus, each computing node can include 8 GPUs, and the 8 computing nodes can have a total of 64 GPUs. Compared with embodiment 3, the processor density is doubled.

[0120] Understandably, since each compute node's connector outputs 32 channels, which is double the number in the solution in Example 3, each switch receives a total of 32 × 8 = 256 channels of bandwidth from the compute nodes, while the capacity of each switching device remains at 256 channels, which cannot provide additional bandwidth.

[0121] To ensure that each switch can provide additional bandwidth, in some implementations, four 256-channel switching devices (SW1 to SW4) can be set in each switch. SW1 in each switch connects to the corresponding 32×4=128 channels of computing nodes 1 to 4, and SW2 in each switch connects to the corresponding 32×4=128 channels of computing nodes 5 to 8.

[0122] In order to achieve full interconnection between all GPUs, the uplink (connecting compute nodes) of SW1 is a 128-channel signal, while the downlink of SW1 is a 128-channel signal that can be divided into two groups of 64 channels each, which are connected to SW3 and SW4 respectively. Similarly, the downlink of SW2 is also divided into two groups of 128 channels, which are connected to SW3 and SW4 respectively.

[0123] For the bandwidth of SW1 connecting compute nodes 1-4 and the bandwidth of SW2 connecting compute nodes 5-8, SW1 and SW2 are connected via SW3, achieving a 64-channel non-blocking connection; additionally, SW1 and SW2 are connected via SW4, achieving another 64-channel non-blocking connection. Therefore, the uplink bandwidth (128 channels) of either SW1 or SW2 is exactly equal to the bandwidth (128 channels) of the interconnection between SW1 and SW2. Thus, compute nodes 1-8 can achieve a non-blocking full interconnection.

[0124] Secondly, the uplink bandwidth (128 channels) of SW1 connecting compute nodes 1 to 4 is exactly equal to the sum of the downlink bandwidth (64 channels) of SW1 through SW3 and SW4. Therefore, all the bandwidth of compute nodes 1 to 4 can be scaled out of the switch without blocking (1:1 convergence). The same applies to compute nodes 5 to 8 through SW2.

[0125] Among them, SW3 has the same uplink bandwidth as downlink bandwidth (both with 128 channels), and SW4 has the same uplink bandwidth as downlink bandwidth (both with 128 channels). The uplink bandwidth of SW3 and the uplink bandwidth of SW4 are externally extended bandwidths that can be used to achieve interconnection with other computing devices.

[0126] In the above embodiments, by designing a hierarchical network of multiple switching devices within the switch, the design objective of exceeding the bandwidth limit of a single switching device is achieved within the bandwidth limit of each switching device, thus solving the technical problem that PCIe-based supernode AI servers cannot achieve scalability due to bandwidth limitations.

[0127] Based on the content described in the above embodiments, some embodiments of this application also provide a node server, which includes the computing interconnect system described in the above embodiments.

[0128] It should be noted that in the various figures provided in the embodiments of this application, when two connecting lines intersect in the figure, it is assumed that there is no electrical or physical connection between them, unless there is an explicit marking (such as a solid dot).

[0129] Those skilled in the art will also recognize that the modules of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0130] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the technical solutions and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A computing interconnection system, characterized in that, The computing interconnect system includes a computing nodes and m switches, and each computing node includes n processors; a, m, and n are positive integers. The switch includes at least a first switching device and a second switching device. The first switching device is connected to k processors out of the n processors, and the second switching device is connected to the other k processors out of the n processors. The k processors connected to the first switching device in different switches are not identical, and the k processors connected to the second switching device in different switches are not identical; k is a positive integer, and... ; Alternatively, the computing node may further include multiple first rate conversion devices, the input terminals of which are respectively connected to the n processors, and the output terminals of which are connected to the switch. The first rate conversion devices are used to convert the rate of the signal output by the processor from a first rate to a second rate, wherein the second rate is an integer multiple of the first rate.

2. The computing interconnection system according to claim 1, characterized in that, In the case that the switch includes at least a first switching device and a second switching device, the computing node further includes m connectors; the connectors are connected to the n processors; The first switching device in the switch is connected to k processors out of the n processors via its corresponding connector, and the second switching device in the switch is connected to the other k processors out of the n processors via its corresponding connector; wherein, the i-th switch out of the m switches is connected to the i-th connector out of the m connectors.

3. The computing interconnection system according to claim 2, characterized in that, The n processors include m processor combinations, and each processor combination includes k processors; wherein the k processors in different processor combinations are not exactly the same. The first switching device in the i-th switch is connected to k processors in the i-th processor combination, and the second switching device in the i-th switch is connected to k processors in the (m-i+1)-th processor combination.

4. The computing interconnection system according to claim 3, characterized in that, The number Q of high-speed serial computer extended bus data transmission channels supported by the first switching device and the second switching device satisfies the following condition: ； Where p represents the minimum bandwidth unit of the processor.

5. The computing interconnect system according to claim 4, characterized in that, The interconnect bandwidth between processors with the same label within different compute nodes is ; And / or, the interconnect bandwidth between processors with different labels within the same compute node is ; And / or, the interconnect bandwidth between processors with different labels within different compute nodes is .

6. The computing interconnect system according to claim 3, characterized in that, The number m of the switches satisfies the following condition: m = C(n, k); Where C(n, k) represents the number of combinations of selecting k processors from n different processors.

7. The computing interconnect system according to claim 6, characterized in that, The total bandwidth of the processor is Where p is the minimum bandwidth unit of the processor, and g satisfies the following condition: 。 8. The computing interconnection system according to claim 2, characterized in that, The computing interconnect system includes at least a first chassis and a second chassis; the first chassis and the second chassis each include m switches; The i-th switch in the first chassis is connected to the i-th switch in the second chassis.

9. The computing interconnect system according to claim 8, characterized in that, The first switching device of the i-th switch in the first chassis is connected to the first switching device of the i-th switch in the second chassis; The second switching device of the i-th switch in the first chassis is connected to the second switching device of the i-th switch in the second chassis.

10. The computing interconnect system according to claim 1, characterized in that, In the case where the computing node includes multiple first rate conversion devices, the computing node also includes multiple connectors; The i-th switch is connected to the output of the i-th first rate conversion device via the i-th connector.

11. The computing interconnect system according to claim 10, characterized in that, The signal output by the processor is a first high-speed serial computer expansion bus signal; The first rate conversion device is used to convert the first high-speed serial computer extended bus signal output by the processor into a second high-speed serial computer extended bus signal, wherein the rate of the second high-speed serial computer extended bus signal is greater than the rate of the first high-speed serial computer extended bus signal.

12. The computing interconnect system according to claim 11, characterized in that, The first rate conversion device is used to receive n1 channels of the first high-speed serial computer expansion bus signals from the n processors, and to output signals to the connector connected thereto. The second high-speed serial computer expansion bus signal; The switch supports n1 high-speed serial computer expansion bus data transmission channels.

13. The computing interconnect system according to claim 11, characterized in that, The switch also includes a second rate conversion device; The second rate conversion device is used to convert the rate of the signal output by the switch from the second rate to the first rate.

14. The computing interconnect system according to claim 13, characterized in that, The switch utilizes the bandwidth saved by the second rate conversion device to enable interconnection with other computing devices.

15. The computing interconnect system according to claim 10, characterized in that, The computing node includes multiple acceleration modules, and each acceleration module includes at least two processors; The i-th connector is connected to the i-th first rate conversion device, and the first rate conversion device is connected to the processor in each of the plurality of acceleration modules.

16. The computing interconnect system according to claim 15, characterized in that, The switch includes a third switching device and a fourth switching device, respectively; The third switching device in the i-th switch is connected to the i-th connector in a / 2 of the a computing nodes, and the fourth switching device in the i-th switch is connected to the remaining... The i-th connector in each computing node is connected.

17. The computing interconnect system according to claim 16, characterized in that, The switch also includes a fifth switching device and a sixth switching device; The input terminal of the fifth switching device is connected to the first output terminal of the third switching device and the first output terminal of the fourth switching device, respectively. The input terminal of the sixth switching device is connected to the second output terminal of the third switching device and the second output terminal of the fourth switching device, respectively.

18. The computing interconnect system according to claim 17, characterized in that, The bandwidth of the first output terminal and the bandwidth of the second output terminal of the third switching device are each half of the uplink bandwidth of the third switching device. The bandwidth of the first output terminal and the bandwidth of the second output terminal of the fourth switching device are each half of the uplink bandwidth of the fourth switching device. The uplink bandwidth and downlink bandwidth of the fifth switching device are the same, and the uplink bandwidth and downlink bandwidth of the sixth switching device are the same; The uplink bandwidth of the fifth switching device and the uplink bandwidth of the sixth switching device are externally extended bandwidths used to achieve interconnection with other computing devices.

19. The computing interconnect system according to claim 1, characterized in that, The switch is a high-speed serial computer expansion bus switch.

20. A node server, characterized in that, The node server includes the computing interconnect system as described in any one of claims 1 to 19.

Images