AI accelerator card performance optimization device and electronic equipment

Abstract

The utility model relates to the technical field of computers, in particular to an AI accelerator card performance optimization device and electronic equipment. The acceleration card is applied to a Feiteng platform and comprises a CPU, a first bridge, a second bridge, a first acceleration card module and a second acceleration card module, one end of the CPU is connected with a first end of the first bridge, a second end of the first bridge is connected with one end of the first acceleration card module, and the first bridge is used for realizing communication between the CPU and the first acceleration card module through the first bridge; the third end of the first bridge is connected with one end of the second bridge, the other end of the second bridge is connected with one end of the second acceleration card module, and the first bridge and the second bridge are connected in series, so that communication between the CPU and the second acceleration card module is achieved. According to the invention, the data transmission rate can be improved, and the data transmission delay can be reduced, so that the requirements of high-performance calculation on data bandwidth and transmission efficiency are met.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to an AI accelerator card performance optimization device and electronic device. Background Technology

[0002] With the rise of high-performance computing fields such as artificial intelligence, the demand for high-bandwidth, low-latency data transmission is becoming increasingly urgent. In recent years, CPU designs for embedded, desktop, and server applications have generally focused on balancing power consumption and performance, rather than pursuing ultimate single-core or multi-core high-performance computing. Taking Phytium CPUs as an example, their design emphasizes versatility and broad application scope, rather than pursuing deep optimization for specific application scenarios. Phytium CPUs are primarily designed for embedded, desktop, and server applications, and their emphasis is not on pursuing high-performance computing. Therefore, when implementing AI accelerator cards on the Phytium platform, their bandwidth performance only reaches the theoretical bandwidth of 0.47Gb / s, far below expectations.

[0003] This phenomenon mainly stems from the limitations imposed on system data transmission by existing design schemes. The traditional parallel connection mode (4-card interconnection) has certain bottlenecks in high-speed data transmission and frequent heterogeneous data interaction, resulting in low data transmission efficiency and restricting the performance release of AI accelerator cards.

[0004] Therefore, how to effectively improve data transmission rate and reduce data transmission latency to meet the requirements of high-performance computing for data bandwidth and transmission efficiency is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this application provides an AI accelerator card performance optimization device and electronic device, which improves data transmission rate and reduces data transmission latency, thereby meeting the requirements of high-performance computing for data bandwidth and transmission efficiency.

[0006] The technical solution adopted by this application to solve its technical problem is:

[0007] In a first aspect, this application provides an AI accelerator card performance optimization device applied to the Phytium platform, including: CPU, first bridge, second bridge, first accelerator card module and second accelerator card module;

[0008] One end of the CPU is connected to the first end of the first bridge, and the second end of the first bridge is connected to one end of the first accelerator card module, which is used to realize communication between the CPU and the first accelerator card module through the first bridge;

[0009] The third end of the first bridge is connected to one end of the second bridge, and the other end of the second bridge is connected to one end of the second accelerator card module, so as to realize communication between the CPU and the second accelerator card module by connecting the first bridge and the second bridge in series.

[0010] Optionally, the memory module is further included; the memory module is connected between the CPU and the first bridge in series.

[0011] Optionally, the memory module includes a plurality of NUMA nodes; the NUMA nodes store local memories corresponding to the CPU.

[0012] Optionally, the first accelerator card module includes four AI accelerator cards.

[0013] Optionally, the second accelerator card module includes four AI accelerator cards.

[0014] Optionally, the CPU and the first bridge, the first bridge and the second bridge, the first bridge and the first accelerator card module, and the second bridge and the second accelerator card module are connected through the PCIe protocol.

[0015] Optionally, the MaxPayLoad value of the PCIe protocol is 256.

[0016] Optionally, the Feiteng platform is a Feiteng 5000C platform.

[0017] In a second aspect, the present application provides an electronic device loaded with the AI accelerator card performance optimization device.

[0018] The present application has the beneficial effect of constructing an efficient AI accelerator card performance optimization device based on a Feiteng platform, which optimizes the traditional parallel connection mode (4-card interconnection) to a series topology mode (8-card interconnection) by adopting a series topology architecture. The device is composed of a CPU, a first bridge, a second bridge, a first accelerator card module, and a second accelerator card module. The CPU communicates with the first accelerator card module through the first bridge, and the first bridge communicates with the second accelerator card module through the second bridge, making data transmission more efficient and smooth. This series architecture can significantly improve data transmission rate and reduce delay compared to the traditional parallel connection mode. Specifically, the four-card interconnection is changed to a new mode of eight-card interconnection, which connects the AI accelerator cards to a single CPU, aiming to improve processing performance, reduce the overhead of cross-CPU communication, and simplify the system architecture, suitable for scenarios requiring high-performance computing and large-scale data processing, such as deep learning training, scientific computing, etc., thereby effectively improving the performance release of the AI accelerator card and meeting the demand for data bandwidth and transmission efficiency of high-performance computing. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a structural schematic diagram of a typical AI acceleration card device provided by the present application;

[0020] Figure 2 is a structural schematic diagram of an AI acceleration card performance optimization device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] The present application will be further described below in conjunction with the drawings and embodiments.

[0022] The concept, specific structure and technical effects of the present application will be clearly and completely described below in conjunction with embodiments and drawings, so as to fully understand the purposes, features and effects of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. In addition, all the coupling / connection relationships involved in the patent do not mean that the components are directly connected, but that a better coupling structure can be composed by adding or reducing coupling accessories according to the specific implementation. The technical features in the creation of the present application can be combined interactively without contradiction and conflict.

[0023] Referring to Figure 1 , Figure 1 is a structural schematic diagram of a typical AI acceleration card device provided by the present application, relates to the field of computer technology and artificial intelligence, and is applied to a Feiteng platform. The device adopts a parallel mode, and the following will be described in detail respectively:

[0024] CPU0 and CPU1 are interconnected to form two groups. In each group, a CPU is connected to a plurality of AI acceleration cards through a bridge SWITCH. When a large amount of data is processed or a complex computing task is processed, in the parallel mode of four-card interconnection, each CPU (such as CPU0 shown in Figure 1 may need to communicate between other CPUs (such as CPU1 shown in Figure 1 ) across nodes to coordinate tasks or share data, and such cross-node communication will increase the delay.

[0025] Based on this, referring to Figure 2 , the structure of the typical AI acceleration card device is adjusted. The device is improved from the original parallel mode (four-card interconnection) to the serial mode (eight-card interconnection) through unified communication of the same CPU, Figure 2 is a structural schematic diagram of an AI acceleration card performance optimization device provided by an embodiment of the present application, which comprises a CPU, a first bridge SWITCH0, a second bridge SWITCH1, a first acceleration card module and a second acceleration card module, and the following will be described in detail respectively:

[0026] One end of the CPU is connected to a first end of the first bridge, a second end of the first bridge is connected to one end of the first acceleration card module, and the communication between the CPU and the first acceleration card module is realized through the first bridge.

[0027] Specifically, the CPU (i.e. Figure 2 The CPU0 shown is connected to the first acceleration card module through the first bridge, wherein the first acceleration card module includes four AI acceleration cards, and the CPU0, the first bridge and the four AI acceleration cards are connected through the PCIe protocol.

[0028] Further, a third end of the first bridge is connected to one end of the second bridge, and the other end of the second bridge is connected to one end of the second acceleration card module, so as to realize the communication between the CPU and the second acceleration card module by connecting the first bridge and the second bridge in series.

[0029] Specifically, compared with the parallel mode in typical applications, the application proposes to form a connection between the first bridge and the second bridge, and the second bridge is also connected to the four AI acceleration cards in the second acceleration card module, so as to form a structure that one CPU (CPU0) is connected through the first bridge and the second bridge, and is interconnected with eight AI acceleration cards. Figure 1 The CPU1 shown can be used to process other data.

[0030] Further, the application further comprises a memory module, and the memory module is connected between the CPU and the first bridge in series.

[0031] Specifically, the CPU in the application adopts the NUMA (Non-Uniform Memory Access, Non-Uniform Memory Access) architecture, which is a memory organization method aimed at improving the performance and scalability of multi-processor systems. In the NUMA architecture, the memory is divided into multiple nodes, and each node contains a certain amount of physical memory. Each CPU is provided with its corresponding local memory, and when the CPU accesses the local memory, the speed is significantly faster than the data transmission that needs to cross the node. However, when the local memory is insufficient, the CPU can access the memory of other nodes, but this will cause higher delay because the data needs to be transmitted between nodes.

[0032] In the embodiment of the application, the memory module includes four NUM nodes: NUM0, NUM1, NUM2 and NUM3.

[0033] Specifically, each NUM node can independently process data, realize parallelization of tasks, and significantly improve overall computing speed. In addition, when processing large-scale data, the data can be divided into different NUM nodes for processing, which can speed up the processing speed of the data. Data segmentation also helps to improve memory utilization efficiency, because each node only needs to process a part of the data.

[0034] Further, the CPU and the first bridge, the first bridge and the second bridge, the first bridge and the first accelerator card module, and the second bridge and the second accelerator card module are connected through the PCIe protocol.

[0035] Specifically, based on the above changes to the connection structure, in order to further improve the performance of the system, it is also necessary to optimize the BIOS (Basic Input / Output System) configuration. The specific operation includes closing smmu (System Management Mode Unit) and configuring MaxPayload value (maximum payload value).

[0036] Among them, BIOS configuration is the first software loaded when the computer starts, which is used to initialize hardware and provide an interface between the operating system and hardware.

[0037] Among them, smmu is a hardware component used to handle system management interrupts and memory protection.

[0038] Among them, MaxPayload value refers to the maximum number of bytes that can be transmitted by PCIe devices in actual use. On the Feiteng 5000C applied in the present application, the MaxPayload value under DevCtl (Device Control) in the node lspci (List PCI Devices) information on the whole link from RC (Root Complex) to EP (Endpoint) is 128. This value determines the maximum number of bytes that can be transmitted by the TLP (Transaction Layer Packet) actually used by the PCIe device (CPU, AI accelerator card). However, the MaxPayload value of the AI accelerator card on the Gen4 (fourth generation) server is usually 256, so the configured MaxPayload value needs to be modified to 256 to further optimize the performance.

[0039] In a second aspect, the present application provides an electronic device loaded with the AI accelerator card performance optimization device described above.

[0040] The foregoing is considered as illustrative only of the principles of the application. Further, various modifications and changes can be made as to the implementation of the present application without departing from the spirit thereof, and it is intended to embrace all such modifications and changes as fall within the scope of the claims.

Claims

1. An AI accelerator card performance optimization device, characterized in that, The application is applied to a Feiteng platform, and comprises a CPU, a first bridge, a second bridge, a first accelerator card module and a second accelerator card module. One end of the CPU is connected to a first end of the first bridge, a second end of the first bridge is connected to one end of the first accelerator card module, and communication between the CPU and the first accelerator card module is realized through the first bridge. A third end of the first bridge is connected to one end of the second bridge, and the other end of the second bridge is connected to one end of the second accelerator card module, so that communication between the CPU and the second accelerator card module is realized through the first bridge and the second bridge in series.

2. The apparatus of claim 1, wherein, Further comprising: a memory module, which is connected in series between the CPU and the first bridge.

3. The apparatus of claim 2, wherein, The memory module comprises a plurality of NUMA nodes, and the NUMA nodes store local memories corresponding to the CPU.

4. The Al accelerator card performance optimization device of claim 1, wherein, The first accelerator card module comprises four AI accelerator cards.

5. The apparatus of claim 4, wherein, The second accelerator card module comprises four AI accelerator cards.

6. The Al accelerator card performance optimization device of claim 1, wherein, The CPU and the first bridge, the first bridge and the second bridge, the first bridge and the first accelerator card module, and the second bridge and the second accelerator card module are connected through a PCIe protocol.

7. The apparatus of claim 6, wherein, The MaxPayLoad value of the PCIe protocol is 256.

8. The Al accelerator card performance optimization device of claim 1, wherein, The Feiteng platform is a Feiteng 5000C platform.

9. An electronic device, comprising: The AI accelerator card performance optimization device is loaded. The application is applied to a Feiteng platform, and comprises a CPU, a first bridge, a second bridge, a first accelerator card module and a second accelerator card module. One end of the CPU is connected to a first end of the first bridge, a second end of the first bridge is connected to one end of the first accelerator card module, and communication between the CPU and the first accelerator card module is realized through the first bridge. A third end of the first bridge is connected to one end of the second bridge, and the other end of the second bridge is connected to one end of the second accelerator card module, so that communication between the CPU and the second accelerator card module is realized through the first bridge and the second bridge in series. Further comprising: a memory module, which is connected in series between the CPU and the first bridge. The memory module comprises a plurality of NUMA nodes, and the NUMA nodes store local memories corresponding to the CPU. The first accelerator card module comprises four AI accelerator cards. The second accelerator card module comprises four AI accelerator cards. The CPU and the first bridge, the first bridge and the second bridge, the first bridge and the first accelerator card module, and the second bridge and the second accelerator card module are connected through a PCIe protocol. The MaxPayLoad value of the PCIe protocol is 256. The Feiteng platform is a Feiteng 5000C platform. The AI accelerator card performance optimization device is loaded. The application is applied to a Feiteng platform, and comprises a CPU, a first bridge, a second bridge, a first accelerator card module and a second accelerator card module. One end of the CPU is connected to a first end of the first bridge, a second end of the first bridge is connected to one end of the first accelerator card module, and communication between the CPU and the first accelerator card module is realized through the first bridge. A third end of the first bridge is connected to one end of

Images