A high-performance computing AI edge server system architecture

Abstract

The present application discloses an AI edge server system architecture with high-performance computing power, which is used to improve the flexibility of use and meet the growing data real-time and security requirements. The AI ​​edge server system architecture in this application includes: a chassis, a CPU computing node module, an accelerator card computing node module, a routing board and a power module. The CPU computing node module includes a PCIe channel, and the accelerator card computing node module is provided with a first accelerator card hot-swappable module. The accelerator card installed on the first accelerator card hot-swappable module is connected to the network. The first accelerator card hot-swappable module is exchanged and converged to the accelerator card computing node module network interface through the switching module. The accelerator card computing node module is also provided with a second accelerator card plug-in module, and the second accelerator card plug-in module is connected to the CPU computing node module through a cable. The accelerator card installed on the second accelerator card plug-in module converges to the accelerator card computing node module network interface through the PCIe channel and the switching module, and the network interface is connected to the routing board network interface.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an AI edge server system architecture with high-performance computing power. Background Art

[0002] With the surge in the number of IoT terminal devices and the growing demand for real-time and security data, edge computing will become crucial in many industry application scenarios, such as road management and autonomous driving in smart transportation, quality inspection and equipment monitoring in smart manufacturing, disease monitoring and auxiliary diagnosis in smart healthcare, etc. Edge computing is still in its early stages of development in China. As data becomes more and more complex, the number of accelerator cards for edge computing servers currently widely used on the market is limited, which will limit the computing power of edge computing.

[0003] At present, in order to solve the problem of limited number of accelerator cards in edge computing servers, a large number of accelerator cards will be expanded to improve the computing power of edge computing. Usually, the expansion is carried out by using the high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIE) expansion method.

[0004] When using the existing acceleration card expansion method, a complex backplane structure is required when expanding the acceleration card. The PCIE bus rate is very high, and many high-speed connectors and signal reconstruction or signal amplification chips are required. Operation and maintenance are difficult, and it is difficult to meet changing needs. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides an AI edge server system architecture with high-performance computing power, which is used to solve the problems of difficult operation and maintenance based on standard servers, limited number of expansion acceleration cards, and limited computing power, while improving the flexibility of use and meeting the growing needs of data real-time and security.

[0006] This application provides an AI edge server system architecture with high-performance computing power, including:

[0007] Chassis, CPU computing node module, accelerator card computing node module, routing board and power supply module;

[0008] The power supply module is electrically connected to the CPU computing node module, the acceleration card computing node module and the routing board respectively;

[0009] The CPU computing node module is installed in the lower layer of the chassis, and the CPU computing node module includes a PCIe channel;

[0010] The acceleration card node module is installed on the upper layer of the chassis, and a first acceleration card hot-swappable module is arranged on the acceleration card computing node module. The acceleration card installed on the first acceleration card hot-swappable module is connected to the network, and a switching module is arranged on the acceleration card. The first acceleration card hot-swappable module exchanges and converges to the network interface of the acceleration card computing node module through the switching module. A second acceleration card plug-in module is also arranged on the acceleration card computing node module, and the second acceleration card plug-in module is connected to the CPU computing node module through a cable. The acceleration card installed on the second acceleration card plug-in module converges to the network interface of the acceleration card computing node module through the PCIe channel and the switching module, and the network interface of the acceleration card computing node module is connected to the network interface of the routing board.

[0011] Optionally, a PCIe slot is provided on the CPU computing node module, and the PCIe slot is used to insert a GPU card.

[0012] Optionally, the CPU computing node module also includes a hard disk module for providing storage function.

[0013] Optionally, the CPU computing node module is also provided with a fan for dissipating heat from the CPU computing node module.

[0014] Optionally, the CPU computing node module is provided with a memory for storing data.

[0015] Optionally, a mid-board is installed on the accelerator card computing node module;

[0016] The network interface of the computing node module of the acceleration card is connected to the middle board through a connector, and the network interface on the middle board is connected to the network cable of the network interface of the routing board.

[0017] Optionally, an OCP3.0 module is set on the CPU computing node module.

[0018] Optionally, the second acceleration card plug-in module includes 18 acceleration card modules supporting PCIe

[0019] Optionally, the chassis is a 4U double-layer chassis.

[0020] Optionally, the chassis further includes a chassis upper cover;

[0021] The chassis upper cover is arranged at the chassis opening and is used for sealing the chassis.

[0022] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0023] In the present application, the CPU computing node module includes a PCIe channel, and the acceleration card computing node module is provided with a first acceleration card hot-swappable module, and the acceleration card installed on the first acceleration card hot-swappable module is connected to the network, and the acceleration card is provided with a switching module, and the first acceleration card hot-swappable module is exchanged and converged at the network interface of the acceleration card computing node module through the switching module, and the acceleration card computing node module is also provided with a second acceleration card plug-in module, and the second acceleration card plug-in module is connected to the CPU computing node module through a cable, and the acceleration card installed on the second acceleration card plug-in module is converged at the network interface of the acceleration card computing node module through the PCIe channel and the switching module, and the network interface of the acceleration card computing node module is connected to the network interface of the routing board. The acceleration card exchanges through the switching module, connects through the network, and converges on the network interface of the routing board. Through the conversion of the routing board, two IPs can be provided to the outside to connect to the terminal. Each acceleration card works independently and can provide multiple signal channels, which greatly improves the AI ​​computing power, improves the flexibility of use, and is simple to operate and maintain, meeting the growing data real-time and security requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 This is a schematic diagram of the structure of the AI ​​edge server system architecture with high performance computing power in this application;

[0026] Figure 2 This is a top view schematic diagram of the CPU computing node module in this application;

[0027] Figure 3 This is a top view schematic diagram of the accelerator card computing node module in this application;

[0028] Figure 4 This is a top view of the accelerator card in this application;

[0029] Figure 5 Schematic diagram of the power supply circuit of the AI ​​edge server system architecture with high-performance computing power in this application. DETAILED DESCRIPTION

[0030] In the present application, the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inside", "outside", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only used to illustrate the relative positional relationships between the various components or components, and do not particularly limit the specific installation orientations of the various components or components.

[0031] In addition, some of the above terms may be used to express other meanings in addition to indicating orientation or positional relationship. For example, the term "on" may also be used to express a certain dependency or connection relationship in some cases. For those of ordinary skill in the art, the specific meanings of these terms in this application can be understood according to specific circumstances.

[0032] In addition, the terms "installed", "set", "provided with", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection, or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be an internal connection between two devices, elements, or components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0033] In addition, the structures, proportions, sizes, etc. drawn in the drawings in the present application are only used to match the contents disclosed in the specification for the technical personnel in this field to understand and read, and are not used to limit the restrictive conditions under which the present application can be implemented. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in the present application without affecting the effects and purposes that can be achieved by the present application.

[0034] The embodiments of the present application provide an AI edge server system architecture with high-performance computing power, which is used to solve the problems of difficult operation and maintenance based on standard servers, limited number of expansion acceleration cards, and limited computing power, while improving the flexibility of use and meeting the growing needs for data real-time and security.

[0035] Typically, edge computing is often associated with the Internet of Things. IoT devices are involved in increasingly powerful processing, so the large amounts of data generated need to be migrated to the "edge" of the network, and the data does not have to be continuously transmitted back and forth between centralized servers for processing. Therefore, edge computing is more efficient in managing large amounts of data from IoT devices, with lower latency, faster processing speeds, and scalability. Edge computing is still in its early stages of development in China. As data becomes more and more complex, the number of accelerator cards for edge computing servers currently widely used on the market is limited, which limits the computing power of edge computing. In the past, if a large number of accelerator cards were to be expanded using the PCIe expansion method, a complex backplane structure would be required. The PCIe bus rate is very high, reaching 2.5Gbps-16Gbps, and requires the use of many high-speed connectors and signal reconstruction or signal amplification chips. Operation and maintenance are relatively difficult, and it is difficult to meet changing needs. The high-performance computing power AI edge server system architecture in this application can effectively solve the above problems.

[0036] See also Figure 1 , Figure 1 This is a schematic diagram of the structure of the AI ​​edge server system architecture with high-performance computing power in this application, including:

[0037] Chassis, CPU computing node module 1, accelerator card computing node module 2, routing board and power supply module 11;

[0038] The power module 11 is electrically connected to the CPU computing node module 1, the accelerator card computing node module 2 and the routing board respectively;

[0039] The CPU computing node module 1 is installed in the lower layer of the chassis, and the CPU computing node module 1 includes a PCIe channel;

[0040] The acceleration card node module is installed on the upper layer of the chassis. The acceleration card computing node module 2 is provided with a first acceleration card hot-swappable module 21. The acceleration card installed on the first acceleration card hot-swappable module 21 is connected to the network. The acceleration card is provided with a switching module 24. The first acceleration card hot-swappable module 21 exchanges and converges on the network interface of the acceleration card computing node module 2 through the switching module 24. The acceleration card computing node module 2 is also provided with a second acceleration card plug-in module. The second acceleration card plug-in module is connected to the CPU computing node module 1 through a cable. The acceleration card installed on the second acceleration card plug-in module converges on the network interface of the acceleration card computing node module 2 through the PCIe channel and the switching module 24. The network interface of the acceleration card computing node module 2 is connected to the network interface of the routing board.

[0041] Among them, as network devices have higher requirements for bandwidth, flexibility and performance, the PCIe standard came into being. PCIe (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard. PCIe belongs to high-speed serial point-to-point dual-channel high-bandwidth transmission. The connected devices are allocated exclusive channel bandwidth and do not share bus bandwidth. It mainly supports active power management, error reporting, end-to-end reliable transmission, hot plugging and service quality and other functions, and has a high data transmission rate. Channel is the path or interface for connecting external signals. PCIe channel is PCIe signal. One channel corresponds to one signal, such as measuring force, temperature, humidity, etc. at multiple points. The signals collected by each channel are generally sent to the signal conditioning circuit in sequence, then A / D conversion, and then transmitted to the microprocessor. The number of channels is usually a multiple of 8, that is, 8 channels, 16 channels, etc.

[0042] Please read further Figure 2 , Figure 3 as well as Figure 4, wherein the power module 11 is connected to the motherboard connector by hot-plugging to supply power to the motherboard. Hot-plugging is hot plugging, which means that the module or board can be inserted or removed from the system without turning off the system power without affecting the normal operation of the system, thereby improving the system's reliability, rapid maintainability, redundancy and timely disaster recovery capabilities. The motherboard is provided with a PCIe slot 13, which is used to insert a GPU card. Generally, three PCIE 5.0 slots are provided, which can be expanded to three A100 GPU cards, and can optimize the utilization of computing resources; the CPU computing node module 1 also includes a hard disk module 12, which is 12 3.5-inch hard disk modules. The backplane of the hard disk module 12 is connected to the slimsas interface cable on the motherboard through the PCIe channel, providing storage function and improving the computer's operating performance and flexibility of use. The CPU computing node module is also provided with a fan 15 for dissipating heat for the CPU computing node module to reduce the decrease in CPU computing performance under high temperature; the CPU computing node module is provided with a memory 14, and the memory is set as 32 DDR5 memories for storing data. The CPU computing node module 1 also includes a motherboard, an OCP3.0 module, 2 Sapphire Rapids series processors, a routing board and a set of 1+1 hot-swappable power modules 11, 1+1 means that each shares 50% of the load. If there is a problem with one power module 11, the other will bear all the loads to prevent the situation where only a single power module 11 fails and causes shutdown. Among them, the first acceleration card computing node module 2 includes two 25 first acceleration card hot-swappable modules in the front, two 16 first acceleration card hot-swappable modules in the rear, including 18 second acceleration card computing node modules 2 using PCIe channels, a total of 100 acceleration cards, and a switching module 24 is set on the acceleration card, and also includes a set of 1+1 hot-swappable power modules 11, and a cooling fan. Optionally, the acceleration card computing node module 2 is also installed with a midboard 23, and the network interface of the acceleration card computing node module 2 is connected to the midboard 23 through a connector, and the network interface on the midboard 23 is connected to the network cable of the network interface of the routing board.

[0043] Please refer to Figure 5, the power supply line scheme can be: 16 first acceleration card hot-swappable modules and 25 first acceleration card hot-swappable modules are directly connected through the network. 18 second acceleration card hot-swappable modules use the PCIe channel on the CPU computing node module motherboard, and the clock sequence number of each channel is different. The power supply of the 18 second acceleration card hot-swappable modules of the acceleration card computing node module 2 is the power supply of the CPU computing node module 1. The 2000W1+1 redundant hot-swappable power supply of the CPU computing node module supplies power to the motherboard, and supplies power to the 18 second acceleration card hot-swappable modules and the fan of the CPU computing node module through cables; the 16 first acceleration card hot-swappable modules and 25 first acceleration card hot-swappable modules of the acceleration card computing node module 2 are directly powered by the above 2000W1+1 hot-swappable power supply.

[0044] The 25 first acceleration card hot-swappable modules and 16 first acceleration card hot-swappable modules of the acceleration card node are directly connected to the network. One switching module can exchange up to 9 acceleration cards. Two groups of 25 first acceleration card hot-swappable modules and two groups of 16 first acceleration card hot-swappable modules are exchanged and converged on 10 network interfaces through 10 switching modules placed on the back of the acceleration card, and are connected to the first middle board through a connector. The 10 network interfaces on the first middle board are connected to the 10 network interfaces of the routing board through network cables; the 18 second acceleration card plug-in modules of the upper-level acceleration card node are connected to the MCIO interface of the main board on the lower-level CPU computing node through the MCIO interface, use PCIe signals, and then exchange and converge to 2 network interfaces through 2 switching modules, and are connected to the second middle board through a connector. The 2 network interfaces on the second middle board are connected to the 2 network interfaces of the routing board through network cables. Finally, all the acceleration cards are converged on the routing board through a total of 12 network interfaces, and the routing board provides 2 IP addresses to the outside to connect to the terminal. Each accelerator card works independently and can provide up to more than 2,000 signal channels, greatly improving AI computing capabilities, increasing flexibility of use, simplifying operation and maintenance, and meeting the growing demands for data real-time and security.

[0045] Optionally, the chassis is a 4U double-layer chassis. "U" in the server field specifically refers to the thickness of a rack-mounted server. It is a unit that represents the external dimensions of a server and is an abbreviation of unit. The detailed dimensions are determined by the American Electronics Industry Association, an industry group. The thickness is in centimeters. 1U is 4.45cm, and 4U is 4 times 1U, which is 17.8cm. In this embodiment, the size of the chassis can be adjusted according to the actual equipment to be placed.

[0046] Optionally, the chassis further includes a chassis upper cover;

[0047] The upper cover of the chassis is arranged at the opening of the chassis to seal the chassis and reduce the possibility of dust entering the chassis and damaging the components in the chassis.

[0048] It should be noted that the above invention content and specific implementation methods are intended to demonstrate the practical application of the technical solution provided in this application and should not be interpreted as limiting the scope of protection of this application. Those skilled in the art can make various modifications, equivalent substitutions or improvements within the spirit and principles of this application. The scope of protection of this application shall be subject to the attached claims.

Claims

1. A high-performance computing AI edge server system architecture, characterized in that: include: Chassis, CPU computing node module, accelerator card computing node module, routing board and power supply module; The power supply module is electrically connected to the CPU computing node module, the acceleration card computing node module and the routing board respectively; The CPU computing node module is installed in the lower layer of the chassis, and the CPU computing node module includes a PCIe channel; The acceleration card computing node module is installed on the upper layer of the chassis, and a first acceleration card hot-swappable module is arranged on the acceleration card computing node module. The acceleration card installed on the first acceleration card hot-swappable module is connected to the network. The acceleration card is provided with a switching module, and the first acceleration card hot-swappable module exchanges and converges the network interface of the acceleration card computing node module through the switching module. The acceleration card computing node module is also provided with a second acceleration card plug-in module, and the second acceleration card plug-in module is connected to the CPU computing node module through a cable, and the acceleration card installed on the second acceleration card plug-in module converges to the network interface of the acceleration card computing node module through the PCIe channel and the switching module, and the network interface of the acceleration card computing node module is connected to the network interface of the routing board; a middle board is installed on the acceleration card computing node module; the network interface of the acceleration card computing node module is connected to the middle board through a connector, and the network interface on the middle board is connected to the network cable of the network interface of the routing board; the second acceleration card plug-in module includes 18 acceleration card modules supporting PCIe.

2. The AI ​​edge server system architecture according to claim 1, characterized in that: The CPU computing node module is provided with a PCIe slot, and the PCIe slot is used to insert a GPU card.

3. The AI ​​edge server system architecture according to claim 1, characterized in that: The CPU computing node module also includes a hard disk module for providing storage function.

4. The AI ​​edge server system architecture according to claim 1, characterized in that: The CPU computing node module is also provided with a fan for dissipating heat from the CPU computing node module.

5. The AI ​​edge server system architecture according to claim 1, characterized in that: The CPU computing node module is provided with a memory for storing data.

6. The AI ​​edge server system architecture according to claim 1, characterized in that: An OCP3.0 module is set on the CPU computing node module.

7. The AI ​​edge server system architecture according to any one of claims 1 to 6, characterized in that: The chassis is a 4U double-layer chassis.

8. The AI ​​edge server system architecture according to claim 7, characterized in that: The chassis also includes a chassis upper cover; The chassis upper cover is arranged at the chassis opening and is used for sealing the chassis.

Images