Acceleration device and server

Abstract

The present application discloses an acceleration device, which relates to the field of computers and is applied to a CPU, including: a GPU controller, a GPU, and an IB switching interface. The GPU controller is connected to the GPU through an NVLink bus, and is connected to the IB switching interface through the IB bus. The CPU can transmit data tasks that need to be processed to the GPU controller through the IB bus through the IB switching interface, and the GPU controller then transmits the data tasks to the GPU through the NVLink bus for data processing. The GPU controller can realize conversion between the IB protocol and the NVLink protocol. This interconnection architecture effectively solves the data transmission bandwidth bottleneck problem of the bidirectional PCIE bus bandwidth of 64GB / s in the existing CPU+GPU architecture, and bandwidth balance is achieved when data is transmitted and exchanged between the CPU memory and the GPU memory.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to an acceleration device for a CPU of a server. Background Art

[0002] The explosive growth of information in modern society has continuously increased the computing power of server hardware, requiring servers to have powerful floating-point operations, matrix operations and large-scale parallel computing capabilities. However, the architecture of the central processing unit (hereinafter referred to as CPU) itself determines that it is more suitable for general computing and has low computing efficiency. The graphics processing unit (hereinafter referred to as GPU) makes up for the disadvantages of general-purpose CPUs in floating-point operations, matrix operations and large-scale parallel computing. The current heterogeneous computing architecture of general-purpose CPU+GPU is an efficient solution for processing large amounts of data processing operations. In the existing server CPU+GPU heterogeneous architecture, the CPU is interconnected with the GPU through the PCI Express (hereinafter referred to as PCIE) bus. PCIE is a high-speed serial point-to-point bus. The PCIE bus can support the configuration of different numbers of lanes, such as x1, x2, x4, x8, and x16, depending on the device bandwidth. Usually, the GPU is interconnected with the CPU through x16 Lane (16 PCIE buses), or the PCIE bus is connected through the PCIE Switch chip to realize the architecture of connecting multiple GPUs under one CPU.

[0003] The CPU and GPU are interconnected via the PCIE bus. The latest PCIE 4.0 rate is 16Gb / s. The total bandwidth of a bidirectionally connected x16 lane PCIE bus is 64GB / s, while the data transmission bandwidth between the CPU and the CPU memory is 500GB / s, and the data transmission bandwidth between the GPU and the GPU memory is 512GB / s. The PCIE data transmission bandwidth between the CPU and the GPU is only about 1 / 8 of the data transmission bandwidth between the CPU and the CPU memory. The bandwidth bottleneck of data transmission between the CPU and the GPU is on the PCIE bus from the CPU to the GPU (or from the CPU to the PCIE switch unit), which greatly limits the bandwidth of data transmission.

[0004] It can be seen that how to solve the bandwidth bottleneck problem of CPU to GPU data transmission is a technical problem that needs to be solved urgently by people in this field. Summary of the invention

[0005] The purpose of this application is to provide an acceleration device that solves the bandwidth bottleneck problem of data transmission from CPU to GPU.

[0006] In order to solve the above technical problems, the present application provides an acceleration device, which is applied to a CPU, comprising:

[0007] GPU controller, GPU, IB switch interface,

[0008] The GPU controller is connected to the GPU via an NVLink bus;

[0009] The GPU controller is connected to the IB switch interface via an IB bus;

[0010] The GPU controller is used to implement the conversion between the IB protocol and the NVLink protocol;

[0011] The CPU is connected to the GPU controller via the IB exchange interface and is used to send data tasks to the GPU controller.

[0012] Preferably, the GPU controller is further used for, after receiving the data task from the CPU through the IB exchange interface, allocating the data task to the GPU and controlling the GPU to perform data processing on the data task.

[0013] Preferably, it also includes: a GPU memory, which is connected to the GPU and is used to cache data of the GPU.

[0014] Preferably, it also includes:

[0015] Management controller, control interface;

[0016] The management controller is connected to the GPU controller and the GPU via a system management bus, and is used to obtain the operating voltage, temperature and power consumption status information of the GPU controller and the GPU;

[0017] The control interface is connected to the management controller via Ethernet;

[0018] The control interface is connected to the CPU and is used to send the operating voltage, temperature and power consumption status information of the GPU controller and the GPU to the CPU.

[0019] Preferably, it also includes: a power supply unit, which is used to supply power to the GPU controller, the GPU, the GPU memory and the management controller.

[0020] Preferably, the GPU controller is a controller based on ARM architecture.

[0021] Preferably, the GPU controller integrates an HBM memory unit internally.

[0022] Preferably, there are multiple GPUs.

[0023] The present application also provides a server, comprising the above-mentioned acceleration device.

[0024] The acceleration device provided in the present application, wherein the wireless broadband technology (InfiniBand, hereinafter referred to as IB), NVLink, is a bus and its communication protocol developed and launched by NVIDIA. The GPU controller is connected to the GPU through the NVLink bus, and is connected to the IB switching interface through the IB bus. The NVLink bus bandwidth can reach 600GB / s. The CPU can transmit the data tasks to be processed to the GPU controller through the IB bus through the IB switching interface, and the GPU controller then transmits the data tasks to the GPU through the NVLink bus for data processing. This interconnection architecture effectively solves the data transmission bandwidth bottleneck problem of the bidirectional PCIE bus bandwidth of 64GB / s in the existing CPU+GPU architecture, and the data is transmitted and exchanged between the CPU memory and the GPU memory to achieve bandwidth balance.

[0025] In addition, the present application also provides a server, including the above-mentioned acceleration device, with the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 This is a schematic diagram of the structure of the acceleration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] The core of this application is to provide an acceleration device to solve the data transmission bandwidth bottleneck problem of 64GB / s of bidirectional PCIE bus bandwidth in the existing CPU+GPU architecture.

[0030] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0031] Figure 1 The schematic diagram of the acceleration device structure provided in the embodiment of the present application is as follows: Figure 1 As shown, this embodiment provides an acceleration device, applied to a CPU, including:

[0032] GPU controller 1, GPU 2, IB switch interface 3,

[0033] GPU controller 1 and GPU2 are connected via NVLink bus;

[0034] The GPU controller 1 is connected to the IB switching interface 3 via an IB bus;

[0035] GPU controller 1 is used to implement the conversion between IB protocol and NVLink protocol;

[0036] The CPU is connected to the GPU controller 1 via the IB exchange interface 3 and is used to send data tasks to the GPU controller 1 .

[0037] The CPU is one of the main devices of an electronic computer and a core component in a computer. Its main function is to interpret computer instructions and process data in computer software. The CPU is the core component in a computer that is responsible for reading instructions, decoding instructions, and executing instructions. The CPU's own architecture determines that it is more suitable for general computing and has low computing efficiency. The GPU is designed based on high throughput and provides an infrastructure for multi-core parallel computing. It has a large number of cores and can support parallel computing of large amounts of data. It has a higher memory access speed and higher floating-point computing capabilities. The heterogeneous computing architecture of CPU+GPU is an efficient solution for processing large amounts of data processing operations.

[0038] It should be noted that NVLink is a bus and its communication protocol developed and launched by NVIDIA. NVLink uses a point-to-point structure and serial transmission, and is used to connect the central processing unit (CPU) and the graphics processing unit (GPU). It can also be used to connect multiple graphics processors. The IB bus has extremely high throughput and extremely low latency, and is used for data interconnection between computers. The IB bus is also used as a direct or switched interconnection between servers and storage systems, as well as an interconnection between storage systems.

[0039] It should be noted that the GPU controller 1 mentioned in this embodiment is a device for implementing conversion between NVLink protocol and IB protocol. This embodiment does not limit the method for the GPU controller 1 to implement protocol conversion, nor does it limit the composition architecture of the GPU controller 1. It can be designed according to specific needs.

[0040] The acceleration device provided in this embodiment connects the GPU controller 1 and the GPU 2 through the NVLink bus, connects the GPU controller 1 and the IB switching interface 3 through the IB bus, and the CPU is connected to the acceleration device through the IB switching interface 3.

[0041] In addition, this embodiment does not limit the power supply method of the acceleration device. For example, the acceleration device is powered by a built-in power supply, or by connecting an external power supply. It can be designed according to specific circumstances.

[0042] Specifically, the CPU can transmit the data tasks that need to be processed to the GPU2 control unit via the IB bus through the IB switching interface 3. The GPU2 control unit transmits the received data tasks to GPU2 and controls GPU2 to process the data. When GPU2 obtains the data processing results, it is transmitted to the CPU via the GPU2 control unit. Due to the high bandwidth of the NVlink bus and the IB bus, the data transmission between the CPU and GPU2 is no longer limited by the PCIE bus bandwidth of 64GB / s, and the data transmission and exchange between the CPU memory and the GPU memory 4 achieves bandwidth balance.

[0043] According to the above embodiment, in order to save CPU computing resources, this embodiment provides a preferred solution, where the GPU controller 1 is also used to assign the data task to GPU2 after receiving the data task of the CPU through the IB exchange interface 3, and control GPU2 to process the data task.

[0044] In the existing CPU+GPU heterogeneous computing architecture, before the CPU transfers the data tasks to be calculated to GPU2, the corresponding allocation and scheduling control work needs to be completed in the CPU.

[0045] Specifically, in the preferred solution provided by this embodiment, the CPU transmits all data tasks to the GPU controller 1, and the GPU controller 1 completes the allocation and scheduling control work that should be performed in the CPU, thereby saving the computing resources of the CPU.

[0046] According to the above embodiment, in order to improve the working efficiency of GPU2 calculation, this embodiment provides a preferred solution, the acceleration device also includes: GPU memory 4, GPU memory 4 is connected to GPU2, and is used to cache data of GPU2.

[0047] The computing power of GPU2 is stronger than that of CPU, but the memory capacity of GPU2 is much smaller than that of CPU. The memory capacity of CPU is generally between 64GB and 256GB, while that of GPU2 is between 4GB and 12GB. Since GPU2 needs to perform a large amount of data calculations, it needs a large enough memory space for data caching. Therefore, before executing the computing task on GPU2, the data required for the computing task can be transferred to GPU memory 4, making full use of the transmission bandwidth between CPU and GPU2, reducing the time spent on data preparation, and improving the computing efficiency of GPU2.

[0048] According to the above embodiment, since the GPU controller 1 and the GPU 2 may affect the working voltage, temperature and power consumption status information of the GPU controller 1 and the GPU 2 due to long working time or large amount of calculation during operation, thereby affecting the service life of the GPU controller 1 and the GPU 2, this embodiment provides a preferred solution, the acceleration device also includes:

[0049] Management controller 5, control interface 6;

[0050] The management controller 5 is connected to the GPU controller 1 and the GPU 2 via a system management bus, and is used to obtain the operating voltage, temperature and power consumption status information of the GPU controller 1 and the GPU 2;

[0051] The control interface 6 is connected to the management controller 5 via Ethernet;

[0052] The control interface 6 is connected to the CPU and is used to send status information of the GPU controller 1 and the GPU 2 to the CPU.

[0053] The management controller 5 mentioned in this embodiment refers to a device for obtaining the working voltage, temperature and power consumption status information of the GPU controller 1 and the GPU 2. This embodiment does not limit the method by which the management controller 5 obtains the working voltage, temperature and power consumption status information of the GPU controller 1 and the GPU 2, nor does it limit the composition architecture of the management controller 5. It can be designed according to specific needs.

[0054] In addition, the management controller 5 obtains the operating voltage, temperature and power consumption status information of the GPU controller 1 and GPU2 by obtaining the information of the voltage detection device, temperature sensor or power consumption detection device provided on the GPU controller 1 and GPU2. Of course, it is not limited to this method, and only provides a preferred implementation scheme.

[0055] System Management Bus (hereinafter referred to as SMBus), SMBus has only two signal lines: a bidirectional data line and a clock signal line. The data transmission rate of SMBus is 100kbps. Although the speed is slow, it has a simple structure and low cost. It is often used to transmit the measurement results of various sensors. Ethernet is a computer local area network technology. It specifies the content including the connection of the physical layer, electronic signals and media access layer protocols. Ethernet has good compatibility, extensive technical support, low cost and high communication rate. Ethernet is currently the most commonly used local area network technology.

[0056] Specifically, the management controller 5 obtains the working voltage, temperature and power consumption status information of the GPU controller 1 and GPU 2 through SMBus, and transmits it to the control interface 6 through Ethernet. The control interface 6 is connected to the CPU, and the CPU receives the status information of the GPU controller 1 and GPU 2. After the CPU obtains the working voltage, temperature and power consumption status information of the GPU controller 1 and GPU 2, it can determine whether the working status of the GPU controller 1 and GPU 2 is normal, and can timely control the GPU controller 1 and GPU 2 to stop working or control the cooling device to cool down. This embodiment does not limit what measures the CPU takes, and it can be set according to actual conditions.

[0057] The management controller 5 receives the operating voltage, temperature and power consumption status information of the GPU controller 1 and the GPU 2. The status information may be monitored in real time or acquired once every preset time. This embodiment does not make any specific requirements.

[0058] The management controller 5 monitors the status information of the GPU controller 1 and GPU2, and transmits the status information to the CPU, which determines whether the working status of the GPU controller 1 and GPU2 is normal, so as to take timely measures to protect the GPU controller 1 and GPU2 and extend their service life.

[0059] According to the above embodiment, this embodiment provides a preferred solution for powering an acceleration device, and the acceleration device also includes: a power supply unit 7, and the power supply unit 7 is used to power the GPU controller 1, GPU2, GPU memory 4 and management controller 5.

[0060] The power supply unit 7 mentioned in this embodiment is responsible for providing power to the acceleration device. The GPU controller 1, GPU 2, GPU memory 4 and management controller 5 need power input when working. This embodiment does not limit the form of the power supply unit 7, which can be set according to specific circumstances. The built-in power supply unit 7 can make the acceleration device more integrated.

[0061] According to the above embodiments, this embodiment provides a preferred solution for the architecture of a GPU controller 1, an acceleration device, and the GPU controller 1 is a controller based on an ARM architecture.

[0062] The processor based on ARM architecture is small in size and can perfectly complete applications in embedded environments. ARM also maintains super performance. The advantage of ARM structure is that it can take into account performance, power consumption, code density, price and other aspects, and it is relatively balanced and has faster instruction execution speed. GPU controller 1 based on ARM architecture is a preferred solution.

[0063] According to the above embodiments, this embodiment provides a preferred solution, in which the GPU controller 1 integrates an HBM memory unit.

[0064] High Bandwidth Memory (HBM) is a high-performance DRAM based on 3D stacking process initiated by AMD and SK Hynix. It is a standardized stacking storage technology that can provide high-bandwidth channels for data within the stack and between storage and logic elements. It is suitable for applications with high memory bandwidth requirements. The GPU controller 1 integrates HBM memory units as data cache units, which is conducive to faster data processing.

[0065] According to the above embodiments, this embodiment provides a preferred solution, where the number of GPU2 is multiple.

[0066] The number of GPU2s connected to the GPU controller 1 can be multiple. Specifically, the CPU transmits the data tasks that need to be calculated to the GPU controller 1 through the IB exchange interface 3. The GPU controller 1 can distribute the data tasks to multiple GPU2s for calculation. Multiple GPU2s process data simultaneously to improve data processing efficiency.

[0067] A server comprises the acceleration device mentioned in any one of the above embodiments.

[0068] The servers mentioned in this embodiment include but are not limited to AI intelligent servers, graphics processing servers and other servers that need to process a large amount of data. The acceleration device is connected to the CPU of the server, and the CPU transmits the tasks that need to be processed to GPU2 through GPU controller 1. Due to the high bandwidth of the NVlink bus and the IB bus, the data transmission between the CPU and GPU2 is no longer limited by the PCIE bus bandwidth of 64GB / s, and the data is transmitted and exchanged between the CPU memory and the GPU memory 4 to achieve bandwidth balance.

[0069] The acceleration device and server provided by the present application are introduced in detail above. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0070] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. An acceleration device, applied to a CPU, characterized in that: include: GPU controller, GPU, IB switch interface, The GPU controller is connected to the GPU via an NVLink bus; The GPU controller is connected to the IB switch interface via an IB bus; The GPU controller is used to implement the conversion between the IB protocol and the NVLink protocol; The CPU is connected to the GPU controller via the IB exchange interface, and is used to send data tasks to the GPU controller; After receiving the data task from the CPU through the IB exchange interface, assigning the data task to the GPU, and controlling the GPU to perform data processing on the data task; Also included: a GPU memory, the GPU memory is connected to the GPU and is used to cache data of the GPU; Also includes: Management controller, control interface; The management controller is connected to the GPU controller and the GPU through a system management bus, and is used to obtain the working voltage, temperature and power consumption status information of the GPU controller and the GPU to monitor the status information of the GPU controller and the GPU, and transmit the status information to the CPU, and the CPU determines whether the working status of the GPU controller and the GPU is normal; The control interface is connected to the management controller via Ethernet; The control interface is connected to the CPU and is used to send the operating voltage, temperature and power consumption status information of the GPU controller and the GPU to the CPU; It also includes: a power supply unit, the power supply unit is used to supply power to the GPU controller, the GPU, the GPU memory and the management controller; The GPU controller is a controller based on ARM architecture; The GPU controller has an internal HBM memory unit; The number of GPUs is multiple.

2. A server, characterized in that: Comprising the acceleration device as claimed in claim 1.

Images