Open accelerator infrastructure server

Abstract

The invention provides an open accelerator infrastructure server which can be applied to the technical field of servers. The server comprises an acceleration node comprising a universal substrate, a plurality of accelerator modules and at least one switch chip, the plurality of accelerator modules and the at least one switch chip being arranged on the universal substrate; the computing node comprises a server main board and a plurality of computing units arranged on the server main board, each computing unit is provided with a network interface card, and the server main board and the universal substrate are arranged in parallel; the front panel is provided with M structure pluggable modules and N network interface cards, the front panel is used for interconnection between the acceleration node and at least one computing node and interconnection between the acceleration node and an acceleration node outside the open accelerator infrastructure server, and the front panel is perpendicular to the universal substrate.

Description

Technical Field

[0001] This application relates to the technical field of servers, and more particularly to an open accelerator infrastructure server. Background Art

[0002] With the accelerating penetration of artificial intelligence technology into all industries, large models have become the core engine driving the intelligent transformation of enterprises. However, the training and deployment of large models pose high requirements for computing power resources, technical thresholds, and operation and maintenance costs. Therefore, large model servers have emerged. Large model servers provide an out-of-the-box, software-hardware collaborative private deployment solution by integrating high-performance computing hardware, optimized large model algorithm frameworks, and one-stop management platforms.

[0003] However, in related technologies, the configuration of large model servers is single and the flexibility is low. Server networking is mainly achieved through expansion at the server dimension, and a large number of cables are required for multi-server interconnection, resulting in high expansion costs and limited application scenarios for the servers. Summary of the Invention

[0004] In view of the above problems, this application provides an open accelerator infrastructure server.

[0005] According to a first aspect of this application, there is provided an open accelerator infrastructure server, including: an acceleration node, including a general substrate, a plurality of accelerator modules, and at least one switching chip, wherein the plurality of accelerator modules and the at least one switching chip are disposed on the general substrate; at least one computing node, including a server motherboard and a plurality of computing units disposed on the server motherboard, each computing unit being configured with a network interface card, wherein the server motherboard is arranged parallel to the general substrate; a front panel, provided with M structure pluggable modules and N network interface cards for the interconnection of the acceleration node and the at least one computing node, and the interconnection of the acceleration node and an acceleration node outside the open accelerator infrastructure server, wherein the front panel is arranged perpendicular to the general substrate, and M and N are integers greater than or equal to 2.

[0006] According to the open accelerator infrastructure server provided by the present application, the acceleration node can be interconnected with at least one computing node through the front panel, and the acceleration node can be interconnected with the acceleration node outside the open accelerator infrastructure server. There are M structural pluggable modules and N network interface cards arranged in the front panel. Compared with the accelerator module expansion achieved from multiple open accelerator infrastructure server dimensions in the related art, since there are M structural pluggable modules and N network interface cards arranged in the front panel, various interconnection schemes can be realized, so that the expansion of the accelerator module can be achieved in a single open accelerator infrastructure server dimension. In addition, the accelerator modules of multiple open accelerator infrastructure servers can be interconnected to form a server cluster, so that the expansion of the accelerator module can be achieved in a single open accelerator infrastructure server dimension and multiple open accelerator infrastructure server dimensions, thereby improving the design flexibility and horizontal bandwidth of the server. Since the distance between the switching chip and the front panel and the accelerator module is relatively close, the cable length is reduced, and the expansion cost is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Through the following description of the embodiments of the present application with reference to the drawings, the above content and other objects, features and advantages of the present application will become clearer. In the drawings:

[0008] Figure 1A Schematically shows the external schematic diagram of a standard open accelerator infrastructure server in the related art;

[0009] Figure 1B Schematically shows Figure 1A The internal schematic diagram of the standard open accelerator infrastructure server shown;

[0010] Figure 1C Schematically shows Figure 1A The system framework diagram of the standard open accelerator infrastructure server shown;

[0011] Figure 2 Schematically shows the structural diagram of the open accelerator infrastructure server according to the embodiment of the present application;

[0012] Figure 3 Schematically shows the structural diagram of the computing node according to the embodiment of the present application;

[0013] Figure 4 Schematically shows the structural diagram of the acceleration node according to the embodiment of the present application;

[0014] Figure 5 Schematically shows the interconnection schematic diagram of the acceleration nodes according to the embodiment of the present application;

[0015] Figure 6Schematically shows a schematic diagram of a front panel according to an embodiment of the present application;

[0016] Figure 7 Schematically shows a schematic diagram of a front panel according to another embodiment of the present application;

[0017] Figure 8 Schematically shows a schematic diagram of a front panel according to yet another embodiment of the present application;

[0018] Figure 9A Schematically shows a schematic diagram of an open accelerator infrastructure server with 16 OAMs according to an embodiment of the present application;

[0019] Figure 9B Schematically shows a schematic diagram of an open accelerator infrastructure server with 8 OAMs and 8 NICs according to an embodiment of the present application;

[0020] Figure 9C Schematically shows a schematic diagram of an open accelerator infrastructure server with 8 OAMs and 24 SSDs according to an embodiment of the present application;

[0021] Figure 10A Schematically shows an overall block diagram of an open accelerator infrastructure server with 16 OAMs according to an embodiment of the present application;

[0022] Figure 10B Schematically shows an overall block diagram of an open accelerator infrastructure server with 8 OAMs and 8 NICs according to an embodiment of the present application;

[0023] Figure 10C Schematically shows an overall block diagram of an open accelerator infrastructure server with 8 OAMs and 24 SSDs according to an embodiment of the present application. Detailed implementation manners

[0024] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present application. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0025] The terms used herein are merely for describing specific embodiments and are not intended to limit the present application. The terms "including", "comprising", etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0027] In cases where expressions similar to "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0028] In the process of implementing this application, it is found that the traditional cloud computing model in the related art is difficult to meet the real-time inference requirements of high concurrency and low latency. Therefore, the large model server has emerged. By integrating high-performance computing hardware, such as a Graphics Processing Unit (GPU) / Tensor Processing Unit (TPU) cluster, an optimized large model algorithm framework, and a one-stop management platform, it provides an out-of-the-box, software-hardware collaborative privatized deployment solution. The core requirements of the large model server include meeting the enterprise's elastic computing power support for model training and inference, reducing the application threshold of Artificial Intelligence (AI) technology, realizing visual deployment and operation and maintenance, ensuring local data storage and full-link security and controllability, and adapting to the customized needs of multiple scenarios such as energy, manufacturing, and government affairs. The large model server is becoming a key infrastructure for governments and enterprises to promote the large-scale implementation of AI.

[0029] However, the configuration of the large model server in the related art is single and the flexibility is low. It mainly realizes server networking by expanding in the server dimension. A large number of cables are required for multi-server interconnection, resulting in high expansion costs and limited application scenarios for the server.

[0030] Figure 1A Schematically shows an external schematic diagram of a standard open accelerator infrastructure server in the related art. Figure 1B Schematically shows Figure 1A The internal schematic diagram of the standard open accelerator infrastructure server shown. Figure 1C Schematically shows Figure 1A The system framework diagram of the standard open accelerator infrastructure server shown.

[0031] As Figure 1AAs shown, the height of the standard open accelerator infrastructure server of this embodiment is 4 standard rack units (abbreviated as U), which is composed of 1 CPU node with a height of 1U and 1 GPU node with a height of 3U. The GPU node has 8 accelerator modules (OAM). The front panel includes the front panel 110 of the CPU node and the front panel 120 of the GPU node. The typical configuration is that the front panel 110 of the CPU node is configured with 2 network interface card (NIC) modules, and the NIC modules can be connected to the external service plane switch through a direct attach cable (DAC) or an active optical cable (AOC). The front panel 120 of the GPU node is configured with 4 NIC modules and 8 quad small form factor pluggable - double density (QSFP - DD) modules. The NIC modules can be used to connect to the external parameter plane switch, and the QSFP - DD interface modules can be interconnected with the QSFP - DD interface modules of other GPU nodes through DAC / AOC cables to realize the interconnection networking of a larger - scale accelerator module.

[0032] As Figure 1B shown, the GPU node 120 can be composed of OAM and a universal baseboard (UBB). 8 OAMs are arranged on the UBB, and the 8 OAMs are arranged on the top surface of the UBB. This GPU node 120 is located at the lower 3U position of the standard open accelerator infrastructure server. The upper 1U of the standard open accelerator infrastructure server is the CPU node 110. The CPU node 110 can include a server motherboard and two computing units. The UBB and the server motherboard are arranged in parallel. At the same time, a host interface board (HIB) 130 is vertically arranged at the rear side of the CPU node 110 and the GPU node 120. The CPU node 110 and the GPU node 120 are respectively connected by plugging in two backplane connectors 131 of the HIB "130", that is, the CPU node 110 and the GPU node 120 are interconnected through the HIB "130" bridge.

[0033] As Figure 1CAs shown, the HIB "130" incorporates 4 high-speed serial computer expansion bus switch chips (Peripheral Component Interconnect Express Switch, abbreviated as PCIE switch chip), namely SW1, SW2, SW3, and SW4. The PCIE switch chips are mainly used to expand the connection to multiple OAMs. The 4 PCIE switch chips SW1, SW2, SW3, and SW4 are connected in sequence. Among them, the PCIE switch chip SW1 and the PCIE switch chip SW4 respectively provide a group of 16-channel 5th-generation PCIE (abbreviated as PCIE GEN5 X16) as the upstream interface, which is connected to the computing units CPU0 and CPU1 of the CPU node 110. Since the PCIE switch chips are built into the HIB "130", the PCIE switch chips SW1, SW2, SW3, and SW4 are relatively far from the 8 OAMs, NIC modules, and QSFP-DD interface modules. Therefore, retimers are required for the connection between the PCIE switch chips SW1, SW2, SW3, and SW4 and the 8 OAMs, NIC modules, and QSFP-DD interface modules to improve the performance of the open accelerator infrastructure. In Figure 1C a double backslash "\\" is used to represent the retimer in. Each PCIE switch chip simultaneously provides two downstream ports to connect to the OAMs and one downstream port to connect to the NIC module. The NIC module is used to realize the horizontal expansion interconnection of the OAMs between different server racks. The 8 OAMs arranged in the UBB of the GPU node 120 support full interconnection. At the same time, each OAM provides a group of x4 serializer and serializer (abbreviated as Serdes) to connect to the QSFP-DD interface module on the front panel, and the vertical expansion interconnection of the OAMs between different GPU nodes and different servers can be realized through the QSFP-DD interface.

[0034] From the above Figures 1A - 1CIt can be known that in the standard open accelerator infrastructure server in the related art, due to the limited number of Serdes in a single OAM within the UBB, some Serdes of the OAM are used for OAM interconnection within the GPU node through on-board routing, and the remaining Serdes are connected to the QSFP-DD interface. Although the OAM vertical expansion interconnection between different GPU nodes is achieved through the QSFP-DD interface, since it provides a connection of x4 Serdes, the bandwidth can reach 448 Gb / s in the case of high bandwidth, and generally can only reach 224 Gb / s bandwidth. At the same time, the OAM interconnection bandwidth between GPU nodes is limited, and the OAM interconnection bandwidth within the GPU node also fails to adopt the maximum bandwidth design, resulting in no computing power advantage in some deep training fields. At the same time, when using the QSFP-DD interface to expand the OAM interconnection of multiple GPU nodes, a large number of QSFP-DD cables are required, and an additional Ethernet vertical expansion switch needs to be configured, resulting in high expansion costs. Using HIB to implement the interconnection between the CPU node and the GPU node makes the distance between the switching chip and devices such as the front panel and OAM relatively far, and a retimer is required to improve the signal integrity performance of the server. Moreover, the fixed QSFP-DD interface on the front panel limits the front panel from adopting diverse configuration schemes, resulting in limited application scenarios.

[0035] In view of this, an embodiment of the present application provides an open accelerator infrastructure server, including: an acceleration node, including a general substrate, a plurality of accelerator modules, and at least one switching chip, wherein the plurality of accelerator modules and at least one switching chip are arranged on the general substrate; at least one computing node, including a server motherboard and a plurality of computing units arranged on the server motherboard, each computing unit is configured with a network interface card, wherein the server motherboard is arranged parallel to the general substrate; a front panel, provided with M structure pluggable modules and N network interface cards, the front panel is used for the interconnection between the acceleration node and at least one computing node, and the interconnection between the acceleration node and an acceleration node outside the open accelerator infrastructure server, wherein the front panel is arranged perpendicular to the general substrate, and M and N are integers greater than or equal to 2.

[0036] Figure 2 The structural diagram of the open accelerator infrastructure server according to an embodiment of the present application is schematically shown.

[0037] As Figure 2As shown in the figure, the open accelerator infrastructure server 200 of this embodiment includes: an acceleration node 210, which includes a general substrate, a plurality of accelerator modules, and at least one switching core; at least one computing node 220, which includes a server motherboard and a plurality of computing units arranged on the server motherboard, and each computing unit is configured with a network interface card; a front panel 230, which is provided with M structure pluggable modules and N network interface cards. The front panel 230 is used for the interconnection of the acceleration node and at least one computing node, and the interconnection of the acceleration node and the acceleration node outside the open accelerator infrastructure server. M and N are integers greater than or equal to 2.

[0038] According to an embodiment of the present application, an open accelerator infrastructure server (abbreviated as OAI server) is a server architecture designed based on open standards, aiming to optimize the deployment and management of artificial intelligence hardware in a data center. By defining and implementing a series of standards, AI accelerator modules from different manufacturers can be seamlessly integrated into a unified computing platform, thereby improving the hardware compatibility and interoperability.

[0039] According to an embodiment of the present application, the acceleration node is the core of the OAI server, responsible for managing task scheduling, data preprocessing and postprocessing, and system-level coordination work. In the OAI server, the accelerator module is also the accelerator module of the open computing project. The acceleration node is also called a GPU node, and can include a general substrate, a plurality of accelerator modules, and at least one switching chip. Among them, the plurality of accelerator modules and at least one switching chip are arranged on the general substrate. For example, when there are 8 accelerator modules and 4 switching chips, the 8 OAMs and 4 switching chips are all arranged on the general substrate.

[0040] According to an embodiment of the present application, a plurality of computing units can be arranged on the server motherboard. The computing unit can be a central processing unit. The computing node is also called a CPU node, and can include a server motherboard and a plurality of computing units. For example, when the computing node adopts a 2-way CPU design, the computing node includes two CPUs, namely CPU0 and CPU1, and CPU0 and CPU1 can be interconnected through an internal dedicated bus. Two M.2 solid state drives (M.2 SSD) can also be configured inside the computing node as the operating system startup disk of the open accelerator infrastructure server.

[0041] According to an embodiment of the present application, the server motherboard is arranged in parallel with the general substrate, and the front panel is arranged perpendicular to the general substrate. The Structured Pluggable Module (400 Gbps Data Fabric Pluggable, CDFP) is a high-performance pluggable optical module mainly used for 400Gbps data transmission in data centers and high-speed networks. The front panel can be used to interconnect the acceleration nodes and the computing nodes. Alternatively, the front panel can be used to interconnect the acceleration nodes, the computing nodes, and the acceleration nodes outside the open accelerator infrastructure server. Alternatively, the front panel can be used to interconnect the acceleration nodes and the acceleration nodes outside the open accelerator infrastructure server. Alternatively, the front panel can be used to interconnect multiple acceleration nodes.

[0042] According to an embodiment of the present application, through the front panel, the acceleration nodes can be interconnected with at least one computing node, and the acceleration nodes can be interconnected with the acceleration nodes outside the open accelerator infrastructure server. There are M Structured Pluggable Modules and N network interface cards arranged in the front panel. Compared with the accelerator module expansion implemented from the dimension of multiple open accelerator infrastructure servers in the related art, since there are M Structured Pluggable Modules and N network interface cards arranged in the front panel, multiple interconnection schemes can be realized, so that the expansion of the accelerator module can be realized in the dimension of a single open accelerator infrastructure server. In addition, the accelerator modules of multiple open accelerator infrastructure servers can be interconnected to form a server cluster, so that the expansion of the accelerator module can be realized in the dimension of a single open accelerator infrastructure server and the dimension of multiple open accelerator infrastructure servers, thereby improving the design flexibility and horizontal bandwidth of the server. Since the distance between the switching chip and the front panel and the accelerator module is relatively close, the cable length is reduced, and the expansion cost is reduced.

[0043] According to an embodiment of the present application, multiple accelerator modules are interconnected with each other; or at least one accelerator module is interconnected with an accelerator module outside the open accelerator infrastructure server to form an open accelerator infrastructure server cluster.

[0044] According to an embodiment of the present application, the acceleration node(s) may include one or more.

[0045] According to an embodiment of the present application, when the number of acceleration nodes is greater than or equal to 2, multiple acceleration nodes can be interconnected; when the number of acceleration nodes is equal to 1, the acceleration node can be interconnected with an acceleration node outside the open accelerator infrastructure server.

[0046] For example, when the number of acceleration nodes is greater than or equal to 2, multiple acceleration nodes can be interconnected. At the same time, at least one of the multiple acceleration nodes can be connected to an acceleration node outside the open accelerator infrastructure server.

[0047] For example, an open accelerator infrastructure server may include two acceleration nodes, and the two acceleration nodes may be interconnected.

[0048] According to an embodiment of the present application, each acceleration node may include a plurality of accelerator modules, and the plurality of accelerator modules may be interconnected with each other. In addition, at least one accelerator module may also be interconnected with an accelerator module outside the open accelerator infrastructure server to form an open accelerator infrastructure server cluster. For example, each acceleration node may include eight accelerator modules, and the eight accelerator modules may be interconnected with each other. At least one of the eight accelerator modules may also be interconnected with an accelerator module outside the open accelerator infrastructure server to form an open accelerator infrastructure server cluster.

[0049] For example, when the number of acceleration nodes is equal to two, the front panel may interconnect the accelerator modules in the first acceleration node with the switching chip in the first acceleration node to the switching chip for the accelerator modules in the second acceleration node in the second acceleration node. Among them, the switching chip for the accelerator modules in the second acceleration node is interconnected with the accelerator modules in the second acceleration node. Thus, the interconnection between the first acceleration node and the second acceleration node is realized.

[0050] For example, when the number of acceleration nodes is equal to one, the front panel may interconnect the accelerator modules in the acceleration node with the switching chip in the acceleration node to the switching chip for the accelerator modules in the acceleration node outside the open accelerator infrastructure server in the acceleration node outside the open accelerator infrastructure server. Among them, the switching chip for the accelerator modules in the acceleration node outside the open accelerator infrastructure server is interconnected with the accelerator modules in the acceleration node outside the open accelerator infrastructure server. Thus, the interconnection between the acceleration node and the acceleration node outside the open accelerator infrastructure server is realized.

[0051] According to an embodiment of the present application, the accelerator node may also include a network interface card. For example, each accelerator node may include eight accelerator modules and eight network interface cards. This design is applicable to an open accelerator infrastructure server cluster to maximize the point-to-point (Peer TO Peer, P2P) bandwidth of the accelerator modules. In one embodiment, the maximum point-to-point bandwidth of the accelerator modules within the acceleration node can support 8x112G, and the point-to-point bandwidth of the accelerator modules in different acceleration nodes can be 400G.

[0052] According to an embodiment of the present application, by interconnecting multiple acceleration nodes through the front panel and interconnecting the acceleration nodes with acceleration nodes outside the open accelerator infrastructure server, as well as interconnecting multiple accelerator modules in the acceleration nodes with each other and at least one accelerator module with accelerator modules outside the open accelerator infrastructure server, an open accelerator infrastructure server cluster can be realized, thereby improving the design flexibility and horizontal bandwidth of the server, and avoiding the use of multiple cables to connect to implement the server cluster in the related art, reducing the expansion cost.

[0053] According to an embodiment of the present application, the front panel may also be provided with solid-state drives.

[0054] According to an embodiment of the present application, the front panel may be provided with solid-state drives (Solid State Drives, SSDs), structural pluggable modules, and network interface cards according to the layout scheme. For example, in the application scenario of a large model server, the front panel may be provided with structural pluggable modules, network interface cards, and solid-state drives, and the structural pluggable modules and solid-state drives are connected to the acceleration nodes through the structural pluggable module cables.

[0055] The computing unit is electrically connected to at least one of the structural pluggable module or the solid-state drive, and the network interface card is used to connect the computing node to an external service switch.

[0056] According to an embodiment of the present application, there may be multiple computing units. For example, there are 2 computing units in the computing node, namely CPU0 and CPU1.

[0057] According to an embodiment of the present application, the Mini Cool edge IO (MCIO) connector is a high-performance, high-density interface technology designed to meet the high-speed signal transmission requirements of next-generation data centers, servers, and high-performance computing devices. The computing unit can be electrically connected to at least one of the structural pluggable module or the solid-state drive through the MCIO connector.

[0058] In one embodiment, the solid-state drive may include multiple ones. For example, it includes 2 solid-state drives. The structural pluggable module may also include multiple ones. For example, it includes 4 structural pluggable modules. In the case where the computing node includes computing unit CPU0 and computing unit CPU1, computing unit CPU0 may include 4 x16 ports as downlink ports. Computing unit CPU0 is electrically connected to the solid-state drive through one of the downlink ports via the MCIO connector. Computing unit CPU0 may also be electrically connected to the structural pluggable module through other ports among the other downlink ports via the MCIO connector. Computing unit CPU1 may also include 4 x16 ports as downlink ports. Computing unit CPU1 may be electrically connected to the solid-state drive through one of the downlink ports via the MCIO connector. Computing unit CPU1 may also be electrically connected to the structural pluggable module through other ports among the other downlink ports via the MCIO connector. The solid-state drives electrically connected to computing unit CPU0 and computing unit CPU1 may be different. For example, the solid-state drives electrically connected to computing unit CPU0 and computing unit CPU1 are solid-state drive 1 and solid-state drive 2 respectively, and the pluggable modules electrically connected to computing unit CPU0 and computing unit CPU1 may also be different. For example, the pluggable modules electrically connected to computing unit CPU0 may be pluggable module 1 and pluggable module 2, and the pluggable modules electrically connected to computing unit CPU1 may be pluggable module 3 and pluggable module 4.

[0059] In one embodiment, computing unit CPU0 may be electrically connected to 2 M.2 SSDs, and the M.2 SSDs serve as the operating system boot disks of the open accelerator infrastructure server.

[0060] In one embodiment, the network interface card may be used to connect the computing node to an external service switch. Computing unit CPU0 and computing unit CPU1 may also be electrically connected to different network interface cards respectively. For example, computing unit CPU0 may be electrically connected to network interface card NIC0, and computing unit CPU1 may be electrically connected to network interface card NIC1, so as to realize the connection to the external service switch.

[0061] According to an embodiment of the present application, the computing unit in the computing node may be electrically connected to at least one of the structural pluggable module or the solid-state drive, and the network interface card configured in the computing unit may be used to connect the computing node to an external service switch, so as to realize data exchange of the open accelerator infrastructure server cluster.

[0062] Figure 3 The structural diagram of the computing node according to an embodiment of the present application is schematically shown.

[0063] As Figure 3As shown, the computing node 220 adopts a dual-CPU design, that is, it includes a computing unit CPU0 and a computing unit CPU1. The computing unit CPU0 and the computing unit CPU1 are interconnected through an internal dedicated bus. The computing unit CPU0 is electrically connected to an M.2 SSD, and the M.2 SSD serves as the operating system boot disk of the open accelerator infrastructure server. The computing unit CPU0 and the computing unit CPU1 are respectively connected to a fifth-generation double data rate synchronous dynamic random-access memory (Double Data Rate 5 synchronous dynamic random-access memory, abbreviated as DDR5) for storing and transmitting the data processed by the computing units. The computing unit CPU0 and the computing unit CPU1 are also respectively connected to a network interface card, that is, the computing unit CPU0 is electrically connected to the network interface card NIC0, and the computing unit CPU1 is electrically connected to the network interface card NIC1. The network interface card NIC0 and the network interface card NIC1 can be connected to an external service switch through a DAC / AOC cable. The server motherboard of the computing node also includes 8 MCIO connectors for realizing the electrical connection between the computing unit CPU0, the computing unit CPU1 and SSD1, SSD2, CDFP1, CDFP2, CDFP3, CDFP4.

[0064] According to an embodiment of the present application, at least one switching chip is used to electrically connect at least one accelerator module to a connector, and the connector is used to electrically connect the switching chip to the front panel.

[0065] According to an embodiment of the present application, at least one switching chip (Switch, abbreviated as SW) can also be arranged on the general substrate. For example, two switching chips, namely switching chip SW1 and switching chip SW2, can be arranged on the general substrate.

[0066] According to an embodiment of the present application, there can be multiple accelerator modules, and there can also be multiple switching chips. The switching chip can be used to electrically connect at least one accelerator module to a connector. For example, in the case of including 8 accelerator modules and 2 switching chips, the switching chip SW1 and the switching chip SW2 can be respectively electrically connected to 4 accelerator modules, that is, the switching chip SW1 can be electrically connected to the accelerator module OAM1, the accelerator module OAM2, the accelerator module OAM3, and the accelerator module OAM4, and the switching chip SW2 can be electrically connected to the accelerator module OAM5, the accelerator module OAM6, the accelerator module OAM7, and the accelerator module OAM8.

[0067] According to an embodiment of the present application, the connector can be a high-density connector, and the connector is used to electrically connect the switching chip to the front panel. In one embodiment, one switching chip can be electrically connected to multiple high-density connectors (Connector, abbreviated as CONN). For example, the switching chip SW1 can be electrically connected to the high-density connector CONN1, the high-density connector CONN2, the high-density connector CONN3, the high-density connector CONN4, and the high-density connector CONN5, and the switching chip SW2 can be electrically connected to the high-density connector CONN6, the high-density connector CONN7, the high-density connector CONN8, the high-density connector CONN9, and the high-density connector CONN10.

[0068] According to an embodiment of the present application, the switching chip disposed on the general substrate can be used to electrically connect at least one accelerator module to the connector, and the connector is used to electrically connect the switching chip to the front panel, so that the accelerator module can be interconnected with the accelerator module outside the open accelerator infrastructure server to form an open accelerator infrastructure server cluster.

[0069] Figure 4 Schematically shows the structural diagram of the acceleration node according to an embodiment of the present application.

[0070] As Figure 4 shown, the UBB is provided with 2 144-channel fifth-generation PCIE switching chips, namely the switching chip SW1 and the switching chip SW2. The switching chips adopt a balanced topology design (abbreviated as balance topology design). Each switching chip uses 4 x16 ports as the downstream ports, which are respectively connected to 4 OAMs. The switching chip SW1 is connected to OAM1, OAM2, OAM3, and OAM4, and the switching chip SW2 is connected to OAM5, OAM6, OAM7, and OAM8. The remaining 5 x16 ports are connected to the high-density connector CONN. The switching chip SW1 is connected to CONN1, CONN2, CONN3, CONN4, and CONN5, and the switching chip SW2 is connected to CONN6, CONN7, CONN8, CONN9, and CONN10. The switching chip is connected to the front panel through the high-density connector.

[0071] Figure 5 Schematically shows the interconnection schematic diagram of the acceleration node according to an embodiment of the present application.

[0072] As Figure 5As shown in the figure, the UBB of the acceleration node is equipped with 8 OAMs. OAM1, OAM2, OAM3, OAM4, OAM5, OAM6, OAM7, and OAM8 adopt a fully interconnected topology. The interconnection bandwidth of any OAM is 800G, which is compatible with the mainstream OAM in the current market. In addition, this fully interconnected topology no longer expands the QSFP-DD connection externally, but is all used for 8 OAM interconnections, which can maximize the interconnection bandwidth of the 8 OAMs. In the small and medium-sized training models that can be completed by 8 OAMs, increasing the bandwidth can effectively shorten the training time. For the networking interconnection of OAM vertical expansion between acceleration nodes, a switching chip is used to realize OAM interconnection.

[0073] According to an embodiment of the present application, the front panel is further provided with a retimer, which is used to electrically connect the connector with the structural pluggable module.

[0074] In an embodiment, a retimer may be arranged on the front panel, and the retimer is also used to electrically connect the connector with the structural pluggable module.

[0075] Figure 6 The diagram schematically shows a front panel according to an embodiment of the present application.

[0076] like Figure 6 As shown, the front panel 230 is provided with a retimer, and the connector may be a high-density connector. The retimer is used to electrically connect the high-density connector CONN with the structure pluggable module CDFP.

[0077] According to an embodiment of the present application, the front panel is also provided with a double-density four-channel small form-factor pluggable package interface, and the network interface card is used to electrically connect the connector to the double-density four-channel small form-factor pluggable package interface.

[0078] In one embodiment, the front panel may also be provided with a double-density four-channel small form-factor pluggable package interface, and the network interface card may also be used to electrically connect the connector to the double-density four-channel small form-factor pluggable package interface.

[0079] Figure 7 A schematic diagram of a front panel according to yet another embodiment of the present application is schematically shown.

[0080] like Figure 7As shown in the figure, two Quad Small Form-factor Pluggable-Double Density (QSFP-DD) interfaces are arranged on the front panel 230. The Network Interface Card (NIC) electrically connects the high-density connector CONN to the QSFP-DD interface. The NIC can be connected to the switching chip of the UBB through the high-density connector CONN. The NIC can also provide four 112G Ethernet Serdes connections to the two QSFP-DD interfaces. Through this front panel, the GPU node can be connected to the parameter plane switch to form a server cluster. The connection between the NIC, the high-density connector CONN, and the QSFP-DD interface uses the interface technology of 2-channel Ethernet with a transmission rate of 112G (abbreviated as x2 112G ETH), and the connection between the high-density connector CONN and the NIC uses the interface technology of PCIE GEN5X16.

[0081] According to an embodiment of the present application, there are multiple solid-state drives, and at least one solid-state drive is electrically connected to the connector.

[0082] In one embodiment, there can be 4 solid-state drives, and the 4 solid-state drives can be electrically connected to the connector.

[0083] Figure 8 A schematic diagram of the front panel according to another embodiment of the present application is schematically shown.

[0084] As Figure 8 shown, four solid-state drives SSD1, SSD2, SSD3, and SSD4 can be electrically connected to the high-density connector CONN.

[0085] According to an embodiment of the present application, when a retimer is arranged on the front panel, the retimer can be used to electrically connect the connector to the Structured Pluggable Module (SPM). When a QSFP-DD interface is arranged on the front panel, the NIC can be used to electrically connect the connector to the QSFP-DD interface. When the front panel includes multiple solid-state drives, the solid-state drives can be electrically connected to the connector. Thus, there can be multiple layout schemes for the front panel, improving the layout flexibility of the front panel.

[0086] According to an embodiment of the present application, the acceleration node and at least one computing node are interconnected according to the layout scheme.

[0087] According to an embodiment of the present application, there can be multiple layout schemes. For example, the layout scheme can include multiple solid-state drives. For another example, the layout scheme can include a retimer that electrically connects the connector to the SPM. The acceleration node and at least one computing node can be interconnected according to the layout scheme.

[0088] According to an embodiment of the present application, the interconnection between the acceleration node and the computing node can be achieved through different layout schemes of the front panel. A switching chip is arranged on the acceleration node, thereby reducing the distance between the switching chip arranged in the acceleration node and the front panel, and improving the flexibility of the design of the open accelerator infrastructure server.

[0089] According to an embodiment of the present application, it further includes a power distribution unit for supplying power to the computing node and the acceleration node. The preset height of the power distribution unit is 1 standard rack unit, the preset height of the open accelerator infrastructure server is 8 standard rack units, the preset height of the acceleration node is 3 standard rack units, and the preset height of the computing node is 1 standard rack unit.

[0090] According to an embodiment of the present application, the open accelerator infrastructure server may further include a power distribution unit for supplying power to the computing node and the acceleration node.

[0091] According to an embodiment of the present application, the preset height of the open accelerator infrastructure server is 8 standard rack units (abbreviated as 8U), the preset height of the power distribution unit is 1 standard rack unit (abbreviated as 1U), the preset height of the acceleration node is 3 standard rack units (abbreviated as 3U), and the preset height of the computing node is 1 standard rack unit (abbreviated as 1U).

[0092] According to an embodiment of the present application, since the open accelerator infrastructure server is 8U high, the acceleration node is 3U high, the power distribution unit is 1U high, the computing node is 1U high, and the open accelerator infrastructure server includes at least one 1U-high computing node and one 1U-high power distribution unit, therefore, the configurable height in the open accelerator infrastructure server is still 6U high.

[0093] In one embodiment, for the remaining 6U high in the open accelerator infrastructure server, it can be configured as two 3U-high acceleration nodes. In one embodiment, for the remaining 6U high in the open accelerator infrastructure server, it can be configured as one 1U-high computing node and one 3U-high acceleration node, and the remaining 2U high space can be not configured or configured with less than or equal to two 1U-high computing nodes. In one embodiment, for the remaining 6U high in the open accelerator infrastructure server, it can be configured as two 3U-high acceleration nodes.

[0094] According to an embodiment of the present application, when the 8U-high open accelerator infrastructure server is configured with a basic 1U-high power distribution unit and a 1U-high computing node, there is still 6U high space remaining. For the remaining 6U high space, it can be flexibly configured, improving the flexibility of the node configuration of the open accelerator infrastructure server.

[0095] According to an embodiment of the present application, the layout plan includes the number of solid-state drives, the number of structural pluggable modules, and the number of network interface cards determined according to the number of acceleration nodes and the number of computing nodes.

[0096] According to an embodiment of the present application, the configuration constraint conditions are determined according to the preset height of the open accelerator infrastructure server, the preset height of the acceleration nodes, and the preset height of the computing nodes. When the preset height of the open accelerator infrastructure server, the preset height of the acceleration nodes, and the preset height of the computing nodes are determined, the configuration constraint conditions can be obtained. That is, for example, when the preset height of the open accelerator infrastructure server is 8U, the preset height of the power distribution unit is 1U, the preset height of the acceleration nodes is 3U, the preset height of the computing nodes is 1U, and the open accelerator infrastructure server includes at least one 1U-high computing node and one 1U-high power distribution unit, the configuration constraint condition obtained is that the height of the configured nodes is less than or equal to 6U.

[0097] According to an embodiment of the present application, when the configuration constraint conditions are determined, the nodes of the open accelerator infrastructure server can be configured according to the preset requirements of the open accelerator infrastructure server and the configuration constraint conditions, so as to obtain the node configuration plan of the open accelerator infrastructure server. Different node configuration plans can support three types of models, namely, 8OAM servers, 16OAM servers, and general computing servers.

[0098] According to an embodiment of the present application, when the node configuration plan is determined, the number of solid-state drives, the number of structural pluggable modules, and the number of network interface cards can be determined through the number of acceleration nodes and the number of computing nodes, thereby constituting a layout plan. Different numbers of solid-state drives, different numbers of structural pluggable modules, and different numbers of network interface cards can constitute different layout plans. For example, when the node configuration plan is determined, that is, when the open accelerator infrastructure server includes a 1U-high power distribution unit, two 3U-high acceleration nodes, and one 1U-high computing node, the number of solid-state drives can be 0, the number of network interface cards in the computing node can be 2, the number of structural pluggable modules in the computing node can be 8, the number of network interface cards in the acceleration node can be 2, and the number of structural pluggable modules in the computing node can be 4, thereby constituting a layout plan.

[0099] According to an embodiment of the present application, by changing the number of network interface cards and structural pluggable modules in the nodes, different layout plans can be constituted, thereby improving the flexibility of the layout plan, the design flexibility of the server, and the horizontal bandwidth, and reducing the expansion cost.

[0100] Figure 9ASchematic diagram showing an open accelerator infrastructure server with 16 OAMs according to an embodiment of the present application. Figure 9B Schematic diagram showing an open accelerator infrastructure server with 8 OAMs and 8 NICs according to an embodiment of the present application. Figure 9C Schematic diagram showing an open accelerator infrastructure server with 8 OAMs and 24 SSDs according to an embodiment of the present application.

[0101] As Figures 9A - 9C shown, the open accelerator infrastructure servers are all 8U-high open accelerator infrastructure servers, which can adopt an integrated chassis design, be compatible with 19-inch and 21-inch cabinets, and the open accelerator infrastructure server is composed of a 1U-high power distribution unit 930, two 3U-high acceleration nodes 910, and one 1U-high computing node 920.

[0102] As Figure 9A shown, 12 structural pluggable modules and 4 network interface cards can be arranged on the front panel of the open accelerator infrastructure server. The front panel can include the front panel 910 of the acceleration node and the front panel 920 of the computing node.

[0103] The front panel 910 of the acceleration node of the open accelerator infrastructure server is composed of 2 NICs "901" and 8 CDFPs "902", and the front panel 920 of the computing node is composed of 2 NICs "901" and 4 CDFPs "902". Since there are many CDFPs "902" on the front panel of the open accelerator infrastructure server, the 8 OAMs of the upper 3U-high acceleration node can be interconnected with the 8 OAMs of the lower 3U-high acceleration node. That is, the open accelerator infrastructure server is a server with 16 interconnected OAMs.

[0104] As Figure 9B shown, 6 structural pluggable modules and 10 network interface cards can be arranged on the front panel of the open accelerator infrastructure server. The front panel can include the front panel 910 of the acceleration node and the front panel 920 of the computing node.

[0105] The front panel 910 of the acceleration node of the open accelerator infrastructure server is composed of 8 NICs "901" and 2 CDFPs "902", and the front panel 920 of the computing node is composed of 2 NICs "901" and 4 CDFPs "902". The 8 OAMs of the upper 3U-high acceleration node and the 8 OAMs of the lower 3U-high acceleration node of the open accelerator infrastructure server are independent of each other. Therefore, the open accelerator infrastructure server is a server based on 8 OAMs and 8 NICs.

[0106] As Figure 9CAs shown, the front panel of the open accelerator infrastructure server can be equipped with 6 structurally pluggable modules, 4 network interface cards, and 24 solid-state drives. The front panel can include the front panel 910 of the acceleration node and the front panel 920 of the computing node.

[0107] The front panel 910 of the acceleration node of the open accelerator infrastructure server consists of 2 NICs "901", 2 CDFPs "902", and 24 SSDs "903". The front panel 920 of the computing node consists of 2 NICs and 4 CDFPs "902". The open accelerator infrastructure server is formed by replacing the 6 NICs of the acceleration node in the server shown above with 24 SSDs "902". That is, the 8 OAMs of the upper 3U-high acceleration node and the 8 OAMs of the lower 3U-high acceleration node of the open accelerator infrastructure server are also independent of each other. Therefore, the open accelerator infrastructure server is a server based on 8 OAMs and 24 SSDs. Figure 9C As shown, the open accelerator infrastructure server is formed by replacing the 6 NICs of the acceleration node in the server shown above with 24 SSDs "902". That is, the 8 OAMs of the upper 3U-high acceleration node and the 8 OAMs of the lower 3U-high acceleration node of the open accelerator infrastructure server are also independent of each other. Therefore, the open accelerator infrastructure server is a server based on 8 OAMs and 24 SSDs.

[0108] Figure 10A Schematically shows the overall block diagram of an open accelerator infrastructure server with 16 OAMs according to an embodiment of the present application. Figure 10B Schematically shows the overall block diagram of an open accelerator infrastructure server with 8 OAMs and 8 NICs according to an embodiment of the present application. Figure 10C Schematically shows the overall block diagram of an open accelerator infrastructure server with 8 OAMs and 24 SSDs according to an embodiment of the present application.

[0109] As Figure 10A As shown, the front panel of the GPU node is equipped with 2 NIC boards and 8 adapter boards. The 8 OAMs share 2 NIC boards to connect to the parameter plane switch. Two of the 8 adapter boards are used as the upstream interface of the switching chip to connect to the computing node, and the remaining 6 adapter boards are used to form a full-interconnection topology of the 2x2 switching chips of the two GPU nodes, thereby realizing the full interconnection of 16 OAMs. The 16 OAMs complete the vertical expansion networking of OAMs through the switching chip. The interconnection bandwidth of the 16 OAMs can reach 1024 Gbp / s, which is much higher than the bandwidth of 448 Gbp / s or 224 Gbp / s of the vertical expansion networking based on Serdes. Moreover, this full-interconnection topology is not affected by the different Serdes rates of OAMs from different manufacturers and can stably achieve a P2P bandwidth of 1024 Gbp / s. In addition, only by adjusting the wiring method of the CDFP cables between multiple servers, the vertical expansion networking of 32 OAMs or 64 OAMs can also be realized without an external switch. Therefore, this topology can be used to solve the communication bottleneck, effectively reduce the cross-card communication delay, and is suitable for scenarios with large model scale, high computational intensity, and frequent data interaction.

[0110] As Figure 10B shown, each GPU node is equipped with 8 OAMs and 8 NICs. The two GPU nodes are independent. The front panel of the GPU node is equipped with 8 NIC boards and 2 adapter boards. This topology is suitable for OAM cluster networking, and the P2P bandwidth of OAM can achieve the maximum design. Among them, the P2P bandwidth within the node is the fully interconnected bandwidth, with a maximum support of 8x112G, and the P2P bandwidth between different nodes is 400G.

[0111] As Figure 10C shown, the front panel of the GPU node is equipped with 2 NIC boards, 2 adapter boards, and 6 SSD boards. The SSD boards are connected to the downstream ports of the switching chip through cables, and each switching chip can be expanded downstream by 12 SSD boards. It can support applications with direct GPU storage, and has obvious advantages in application scenarios such as generative AI and multi-modal that have high data processing overhead.

[0112] As Figures 10A - 10C shown, the main difference between them lies in the different connection methods to the switching chip. Figures 10A - 10C The adapter boards H1~H4 and F1~F12 in [[ ]] are adapter boards related to CDFP, the NIC boards N1~N16 are adapter boards related to NIC, and the SSD boards S1~S12 are adapter boards related to SSD. Moreover, one SSD board can represent 4 SSDs.

[0113] According to the embodiments of the present application, in different usage scenarios, the GPU node can flexibly arrange the number of NICs, with a maximum of 8 NICs. Each OAM can exclusively enjoy one NIC bandwidth. Compared with the related technology where 4 NIC bandwidths need to be allocated to 8 OAMs, the OAM interconnection bandwidth within a single acceleration node and the OAM interconnection bandwidth between different acceleration nodes are improved. In addition, the open accelerator infrastructure server can be deployed privately on a single machine for large models with hundreds of billions of parameters. Compared with the related technology, there is no need to deploy a multi-server cluster, with low hardware and operation and maintenance costs, local data processing, which can effectively ensure data security, and has the characteristics of high performance, low cost, high security, and fast deployment, making the open accelerator infrastructure server have advantages in the field of large models such as deep training.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in an order different from that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0115] Those skilled in the art will appreciate that the features described in the various embodiments of the present application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments of the present application can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present application.

[0116] The embodiments of the present application have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present application. Although the embodiments have been described separately above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of the present application, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present application.

Claims

1. An open accelerator infrastructure server, characterized in that: include: An acceleration node, comprising a universal substrate, a plurality of accelerator modules and at least one switching chip, wherein the plurality of accelerator modules and the at least one switching chip are arranged on the universal substrate; at least one computing node, comprising a server mainboard and a plurality of computing units arranged on the server mainboard, each of the computing units being configured with a network interface card, wherein the server mainboard is arranged in parallel with the universal substrate; A front panel is provided with M structural pluggable modules and N network interface cards, and the front panel is used for interconnecting the acceleration node with the at least one computing node, and interconnecting the acceleration node with an acceleration node outside the open accelerator infrastructure server, wherein the front panel is vertically arranged with the universal substrate, and M and N are integers greater than or equal to 2.

2. The server according to claim 1, characterized in that: The multiple accelerator modules are interconnected with each other; or at least one of the accelerator modules is interconnected with an accelerator module outside the open accelerator infrastructure server to form an open accelerator infrastructure server cluster.

3. The server according to claim 2, characterized in that: The front panel is also provided with: a solid state hard disk; The computing unit is electrically connected to at least one of the structural pluggable module or the solid state hard disk, and the network interface card is used to connect the computing node to an external service switch.

4. The server according to claim 3, characterized in that: The at least one switching chip is used to electrically connect at least one of the accelerator modules to a connector, and the connector is used to electrically connect the switching chip to the front panel.

5. The server according to claim 4, characterized in that: The front panel is also provided with a retimer, and the retimer is used to electrically connect the connector with the structural pluggable module.

6. The server according to claim 4, characterized in that: The front panel is also provided with a double-density four-channel small form-factor pluggable package interface, and the network interface card is used to electrically connect the connector with the double-density four-channel small form-factor pluggable package interface.

7. The server according to claim 4, characterized in that: The solid state hard disk includes a plurality of solid state hard disks, and at least one of the solid state hard disks is electrically connected to the connector.

8. The server according to claim 1 or 2, characterized in that: When the number of the acceleration nodes is greater than or equal to 2, the plurality of acceleration nodes are interconnected; When the number of the acceleration nodes is equal to 1, the acceleration node is interconnected with the acceleration node outside the open accelerator infrastructure server.

9. The server according to claim 3, characterized in that: It also includes a power distribution unit for supplying power to the computing node and the acceleration node, the preset height of the power distribution unit is 1 standard rack unit, the preset height of the open accelerator infrastructure server is 8 standard rack units, the preset height of the acceleration node is 3 standard rack units, and the preset height of the computing node is 1 standard rack unit.

10. The server according to claim 9, characterized in that The front panel is provided with 12 structural pluggable modules and 4 network interface cards; Alternatively, the front panel is provided with 6 of the structural pluggable modules and 10 of the network interface cards; Alternatively, the front panel is provided with 6 of the structural pluggable modules, 4 of the network interface cards and 24 of the solid state hard disks.

Images