Heterogeneous high-performance universal deep learning edge computing platform

Abstract

The invention relates to a heterogeneous high-performance general deep learning edge computing platform, which comprises a hardware platform, and is characterized in that the hardware platform comprises a front panel, a host and a rear panel; the host comprises a main board, an operating system, slave equipment and a dual power supply; the main board adopts a Feiteng host CPU (Central Processing Unit); the slave equipment can be simultaneously equipped with a mercuric chloride 310B module, a mercuric chloride 310P module and a day number intelligent core V100 module; the slave device is communicated with the main board, the main board can provide two paths of X8 and one path of X4 PCIE channels, the first path of X8 channel is a CPU direct-out channel, the other path of X8 channel and the one path of X4 channel are PCIE bridge piece switching output, the CPU direct-out channel is allocated to a day number intelligent core V100 board card with the highest performance, the second path of X8 channel is allocated to a Huawei mercuric chloride 310P board card, the X4 channel is allocated to a Huawei mercuric chloride 310B board card, and the CPU direct-out channel is allocated to a Huawei mercuric chloride 310B board card. The method is used in the EP mode. According to the invention, complex and changeable deep learning tasks can be supported.

Description

Technical Field

[0001] The present invention belongs to the field of edge intelligent computing technology and relates to a deep learning edge computing platform, in particular to a heterogeneous high-performance general deep learning edge computing platform. Background Art

[0002] In today's digital transformation, deep learning technology has attracted much attention due to its outstanding performance in image recognition, natural language processing, speech recognition and other fields. More and more data needs to be processed and analyzed in real time at the edge, which has given rise to the demand for high-performance, low-latency and high-efficiency computing capabilities. Due to factors such as network transmission delays, traditional data center or cloud processing methods can no longer meet the needs of applications with high real-time requirements in some scenarios. Traditional edge computing devices have low efficiency in multi-modal data processing due to their single computing resources and cannot meet the needs of multi-scenario applications.

[0003] Therefore, without solving the above problems, the present invention proposes a heterogeneous high-performance general-purpose deep learning edge computing platform.

[0004] After searching, no public documents of the prior art identical or similar to the present invention were found. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a heterogeneous high-performance general-purpose deep learning edge computing platform that can support complex and varied deep learning tasks.

[0006] The present invention solves the practical problem by adopting the following technical solutions:

[0007] A heterogeneous high-performance general-purpose deep learning edge computing platform, comprising: a hardware platform, the hardware platform comprising: a front panel, a host and a rear panel;

[0008] The host includes: a mainboard, an operating system, a slave device and a dual power supply; the mainboard adopts the host CPU of Feiteng D2000; the slave device can be equipped with Ascend 310B, Ascend 310P, and Tianshu Zhixin V100 modules at the same time; the slave device communicates with the mainboard through the PCIE bus, wherein the mainboard can provide two X8 and one X4 PCIE channels, the first X8 channel is CPU direct output, the other X8 channel and one X4 channel are PCIE bridge chip transfer outputs, the CPU direct output channel is allocated to the highest performance Tianshu Zhixin V100 board, the second X8 channel is allocated to the Huawei Ascend 310P board, and the X4 channel is allocated to the Huawei Ascend 310B board for use in EP mode.

[0009] Furthermore, the front panel includes an indicator light, a USB interface, a VGA interface, a reset switch and a PS2 interface related to the mainboard.

[0010] Moreover, the rear panel includes audio interface, serial port, display interface and network interface related to the motherboard, as well as peripheral interfaces such as USB interface, HDMI interface, Gigabit network port and serial port in the RC mode of the Ascend 310B module.

[0011] Moreover, the external device data first obtains image data, audio data or other external data through the host CPU, and then performs relevant preprocessing according to application requirements. Subsequently, the data is distributed to each accelerator card through the PCIE bus for hardware encoding and decoding and neural network calculations. The CPU's time-consuming calculations are completed in a short time within the NPU, and then returned to the CPU through the PCIE bus for subsequent data integration and output display.

[0012] Moreover, the overall software logical structure of the artificial intelligence computing system of the platform includes: the underlying hardware layer, the middle layer, the pooling layer and the business layer; the underlying hardware layer needs to adapt the interfaces of each intelligent computing module to the corresponding functional interfaces of the Feiteng D2000 host module according to the above-mentioned hardware platform architecture; in the middle layer, exclusive Docker containers for various types of computing modules are constructed, and the containers include computing module drivers, development and reasoning tool chains, and business applications, etc.; in the pooling layer, heterogeneous computing resources are uniformly managed, and a unified programming interface for reasoning applications and a high-performance model reasoning framework are constructed to facilitate the rapid calling and transplantation of business layer applications.

[0013] Moreover, the Docker container needs to be deployed and configured in the physical machine layer and the virtualized container layer respectively; the physical machine layer needs to complete the firmware installation of each intelligent computing module and the adaptation of the underlying driver;

[0014] The specific method is as follows: first, select the driver package and firmware package corresponding to the CPU and operating system kernel version to complete the installation of the underlying driver and firmware. Then, in the virtualization container layer, you first need to create a dedicated Docker container belonging to the module. All subprocesses of each tool chain are run in the container. In the container, all application environments are isolated from each other. The container communication method is used to interact with the software platform, so that the platform can call each development tool chain separately to complete the entire process from data upload to model deployment to program operation; the file system in the Docker container can be recreated or mounted on the host file system, and then the driver of the corresponding module is installed or mounted so that it can be successfully called in the Docker container; finally, the corresponding development tool chain is installed to run the application in the container.

[0015] Moreover, the pooling layer constructs a unified programming interface for heterogeneous platforms, builds a heterogeneous hardware adapter based on the computing interface of Huawei Ascend processors and Tianshu Zhixin processors, and uses hardware device management and model networking functions to connect the programming interface with the inference hardware. A unified programming interface is used at the upper layer to achieve access adaptation of multi-brand and multi-series AI computing chips, and then optimize the software stack according to the performance of each processor.

[0016] Moreover, the pooling layer also builds a multi-model high-performance synchronous reasoning framework, and its specific method is: using the hardware codec of the AI ​​acceleration chip to realize hardware decoding of the input image, reduce the computational load of the CPU, and then unify the preprocessing methods of multiple models; then write the hardware post-processing operator, transfer the post-processing part of the CPU to the NPU for calculation, and finally apply multi-threaded parallel pipeline reasoning technology in the entire processing flow.

[0017] Advantages and beneficial effects of the present invention:

[0018] 1. The present invention proposes a heterogeneous high-performance general-purpose deep learning edge computing platform, which aims to form a flexible and powerful computing architecture by integrating different types of computing resources such as CPU, GPU, NPU and other special accelerators to support complex and changeable deep learning tasks. This platform can not only provide powerful computing performance in hardware, but also use virtualization technology, container technology and high-performance reasoning technology to build a scalable and easy-to-maintain software system architecture, which can dynamically adjust resource allocation according to different application scenarios, effectively manage and schedule these heterogeneous resources, and improve the utilization of computing resources by building a high-performance model reasoning framework, so as to give full play to the computing efficiency of the platform. It solves the technical problem that traditional edge computing devices usually target single modality data and have poor adaptability to multiple types of application scenarios.

[0019] 2. The present invention proposes a heterogeneous high-performance general-purpose deep learning edge computing platform for scenarios such as ships, submarines, and aircraft that require high-density AI computing at the edge. It is based on domestic high-performance CPUs, GPUs, NPUs, FPGAs and other types of heterogeneous computing processors. Through hardware resource virtualization technology, computing resources can be shared and managed by multiple applications or users. Through unified management and scheduling, they are allocated to different applications or users on demand. Combined with the design of a multi-threaded high-performance reasoning framework, the application deployment performance is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the system hardware platform architecture of the present invention;

[0021] Figure 2 It is a schematic diagram of the logical structure of the system software of the present invention;

[0022] Figure 3 It is a schematic diagram of containerized deployment of heterogeneous resources of the present invention;

[0023] Figure 4 This is a schematic diagram of a unified programming interface for heterogeneous platforms of the present invention;

[0024] Figure 5 It is a multi-model synchronous reasoning flow chart of the present invention;

[0025] Figure 6 This is a schematic diagram of multi-threaded parallel pipeline reasoning of the present invention. DETAILED DESCRIPTION

[0026] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings:

[0027] A heterogeneous high-performance general-purpose deep learning edge computing platform, the internal hardware platform architecture diagram of the heterogeneous deep learning computing platform is as follows Figure 1 As shown, it includes: a hardware platform, and the hardware platform includes: a front panel, a host and a rear panel;

[0028] The host includes: a mainboard, an operating system, a slave device and a dual power supply; the mainboard adopts the host CPU of Feiteng D2000; the slave device can be equipped with Ascend 310B, Ascend 310P, and Tianshu Zhixin V100 modules at the same time; the slave device communicates with the mainboard through the PCIE bus, wherein the mainboard can provide two X8 and one X4 PCIE channels, the first X8 channel is CPU direct output, the other X8 channel and one X4 channel are PCIE bridge chip transfer outputs, the CPU direct output channel is allocated to the highest performance Tianshu Zhixin V100 board, the second X8 channel is allocated to the Huawei Ascend 310P board, and the X4 channel is allocated to the Huawei Ascend 310B board for use in EP mode.

[0029] In this embodiment, the front panel includes indicator lights related to the mainboard, a USB interface, a VGA interface, a reset switch and a PS2 interface.

[0030] In this embodiment, the rear panel includes audio interface, serial port, display interface and network interface related to the mainboard, as well as peripheral interfaces such as USB interface, HDMI interface, Gigabit network port and serial port in the RC mode of the Ascend 310B module.

[0031] In this embodiment, the external device data first obtains image data and audio data or other external data through the host CPU, and then performs relevant preprocessing according to application requirements. Subsequently, the data is distributed to each accelerator card through the PCIE bus for hardware encoding and decoding and neural network calculation. The CPU's time-consuming calculations are completed in a short time within the NPU, and then returned to the CPU through the PCIE bus for subsequent data integration and output display.

[0032] The working principle of the system hardware platform is:

[0033] The system motherboard in the host uses the core CPU of Feiteng D2000, and the operating system is Kylin V10; the slave device can be equipped with Ascend 310B, Ascend 310P, and Tianshu Zhixin V100 modules at the same time, and communicate with the motherboard through the PCIE bus, where the motherboard can provide two X8 and one X4 PCIE channels. The first X8 channel is CPU direct output, and the other X8 channel and one X4 channel are PCIE bridge chip conversion outputs. The CPU direct output channel has relatively high performance, and it is allocated to the highest-performance Tianshu Zhixin V100 board. The second X8 channel is allocated to the Huawei Ascend 310P board, and the X4 channel is allocated to the Huawei Ascend 310B board for use in EP mode. The front panel includes indicator lights, USB interface, VGA interface, reset switch and PS2 interface related to the motherboard. The back panel includes audio interface, serial port, display interface and network interface related to the motherboard, as well as peripheral interfaces such as USB interface, HDMI interface, Gigabit network port and serial port in RC mode of Ascend 310B module.

[0034] The external device data first obtains image data, audio data or other external data through the host CPU's network port or USB port, and then performs relevant preprocessing according to application requirements. The data is then distributed to each accelerator card through the PCIE bus for hardware encoding and decoding, neural network calculations, etc. At this stage, the advantages of the artificial intelligence computing card are fully utilized, and the CPU's time-consuming calculations are completed in a short time within the NPU, and then returned to the CPU through the PCIE bus for subsequent data integration, output display, etc.

[0035] In this embodiment, the overall software logical structure of the artificial intelligence computing system of the platform includes: an underlying hardware layer, an intermediate layer, a pooling layer and a business layer; the underlying hardware layer needs to adapt the interfaces of each intelligent computing module to the corresponding functional interfaces of the Feiteng D2000 host module according to the above-mentioned hardware platform architecture; in the intermediate layer, exclusive Docker containers for various types of computing modules are constructed, and the containers include computing module drivers, development and reasoning tool chains, and business applications, etc.; in the pooling layer, heterogeneous computing resources are uniformly managed, and a unified programming interface for reasoning applications and a high-performance model reasoning framework are constructed to facilitate the rapid calling and transplantation of business layer applications.

[0036] In this embodiment, the Docker container needs to perform environment deployment and software configuration from the physical machine layer and the virtualized container layer respectively; the physical machine layer needs to complete the firmware installation of each intelligent computing module and the adaptation of the underlying driver;

[0037] The specific method is as follows: first, select the driver package and firmware package corresponding to the CPU and operating system kernel version to complete the installation of the underlying driver and firmware. Then, in the virtualization container layer, you first need to create a dedicated Docker container belonging to the module. All subprocesses of each tool chain are run in the container. In the container, all application environments are isolated from each other. The container communication method is used to interact with the software platform, so that the platform can call each development tool chain separately to complete the entire process from data upload to model deployment to program operation; the file system in the Docker container can be recreated or mounted on the host file system, and then the driver of the corresponding module is installed or mounted so that it can be successfully called in the Docker container; finally, the corresponding development tool chain is installed to run the application in the container.

[0038] In this embodiment, the pooling layer constructs a unified programming interface for heterogeneous platforms, builds a heterogeneous hardware adapter based on the computing interface of Huawei Ascend processors and Tianshu Zhixin processors, and uses hardware device management and model networking functions to connect the programming interface with the reasoning hardware. A unified programming interface is used at the upper layer to achieve access adaptation of multi-brand and multi-series AI computing chips, and then optimize the software stack according to the performance of each processor.

[0039] In this embodiment, the pooling layer also constructs a multi-model high-performance synchronous reasoning framework, and its specific method is: using the hardware codec of the AI ​​acceleration chip to realize hardware decoding of the input image, reduce the computational load of the CPU, and then unify the preprocessing methods of multiple models; then write the hardware post-processing operator, transfer the post-processing part of the CPU to the NPU for calculation, and finally apply multi-threaded parallel pipeline reasoning technology in the entire processing flow.

[0040] The working principle of the artificial intelligence computing system software of the present invention is:

[0041] The logical structure of the whole software of the artificial intelligence computing system is shown as follows: Figure 2 As shown, the underlying hardware layer needs to adapt the interfaces of each intelligent computing module to the corresponding functional interfaces of the Feiteng D2000 host module according to the above-mentioned hardware platform architecture; build exclusive Docker containers for various types of computing modules in the middle layer, which include computing module drivers, development and reasoning tool chains, and business applications; unify the management of heterogeneous computing resources in the pooling layer, and build a unified programming interface for reasoning applications and a high-performance model reasoning framework to facilitate rapid calling and transplantation of upper-level applications.

[0042] The diagram of containerized deployment of heterogeneous resources in the middle layer is as follows: Figure 3 As shown in the figure, the Docker container needs to perform environment deployment and software configuration from the physical machine layer and the virtualized container layer respectively. The physical machine layer needs to complete the firmware installation of each intelligent computing module and the adaptation of the underlying driver. First, select the driver package and firmware package corresponding to the CPU and operating system kernel version to complete the installation of the underlying driver and firmware. Then, in the virtualized container layer, you first need to create a dedicated Docker container belonging to the module. All subprocesses of each tool chain run in the container. In the container, all application environments are isolated from each other. The container communication method is used to interact with the software platform, so that the platform can call each development tool chain separately to complete the entire process from data upload to model deployment to program operation. The file system in the container can be recreated or mounted on the host file system. Then install or mount the driver of the corresponding module so that it can be successfully called in the Docker container. Finally, install the corresponding development tool chain to run the application in the container. In addition, through the resource restriction function of Docker (such as CPU share and memory restriction), resources are reasonably allocated to avoid resource competition. At the same time, monitoring and log collection tools are integrated into the Docker container to monitor the performance and troubleshoot the containerized heterogeneous modules to ensure system stability. Through the configuration of the Docker container, the running programs of various computing modules can be isolated from each other without affecting each other. Therefore, different businesses based on different intelligent computing modules can be run on the same host at the same time, such as target detection, speech processing, image generation, text error correction, and large model reasoning, which can effectively improve the computing resource utilization of the entire system and the scope of business applications.

[0043] A unified programming interface for heterogeneous platforms is implemented at the pooling layer, as shown in the schematic diagram. Figure 4As shown. A heterogeneous hardware adapter is built based on the computing interface of Huawei Ascend processors and Tianshu Zhixin processors, and the connection between the programming interface and the inference hardware is opened up by using functions such as hardware device management and model networking. A unified programming interface is used at the upper layer to achieve access adaptation of multi-brand and multi-series AI computing chips, and then the software stack is optimized according to the performance of each processor to give full play to the performance of the processor. After building a unified programming interface, multiple types of computing resources are included in the unified resource management pool. The model call can be completed by simply modifying the backend configuration parameter items, and the resources of each module can be uniformly scheduled through the host.

[0044] In the pooling layer, a multi-model high-performance synchronous reasoning framework is constructed. The flowchart is as follows: Figure 5 As shown in the figure, the first step is to use the hardware codec of the AI ​​acceleration chip to implement hardware decoding of the input image and reduce the computing load of the CPU. Then, the preprocessing methods of multiple models are unified. Only one data preprocessing is required to meet the input requirements of multiple models, further reducing the computing load of the CPU. Then, the hardware post-processing operator is written to transfer the post-processing part of the CPU to the NPU for calculation. Finally, multi-threaded parallel pipeline reasoning technology is applied in the entire processing flow, such as Figure 6 As shown, through the pipeline computing processing mode of multiple threads, the utilization rate of the overall platform computing resources is further improved, and the application running performance is improved.

[0045] The innovation of the present invention lies in:

[0046] The innovation of the present invention lies in proposing a heterogeneous high-performance general-purpose deep learning edge computing platform, including a hardware platform and a software framework. The present invention has the following advantages:

[0047] (1) This hardware platform uses three different intelligent computing modules, and completes the driver adaptation of the three different intelligent computing modules Ascend 310B, Ascend 310P, and Tianshu Zhixin V100 modules with the host. It also ensures that the three different intelligent computing modules can run stably and efficiently on the same host at the same time through virtual containerization technology;

[0048] (2) The present invention designs a cross-platform unified programming interface and a multi-model high-performance synchronous reasoning framework, which can incorporate heterogeneous resources into a unified computing resource pool, and manage and schedule them in a unified and effective manner according to application requirements. In addition, by constructing a high-performance model reasoning framework, it can be applied to a variety of deep learning models, improve the utilization rate of computing resources, and give full play to the computing performance of the platform.

[0049] It should be emphasized that the embodiments of the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation modes. Any other implementation modes derived by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.

Claims

1. A heterogeneous high-performance general-purpose deep learning edge computing platform, characterized by: include: A hardware platform, comprising: a front panel, a host and a rear panel; The host includes: a mainboard, an operating system, a slave device and a dual power supply; the mainboard adopts the host CPU of Feiteng D2000; the slave device can be equipped with Ascend 310B, Ascend 310P, and Tianshu Zhixin V100 modules at the same time; the slave device communicates with the mainboard through the PCIE bus, wherein the mainboard can provide two X8 and one X4 PCIE channels, the first X8 channel is CPU direct output, the other X8 channel and one X4 channel are PCIE bridge chip transfer outputs, the CPU direct output channel is allocated to the highest performance Tianshu Zhixin V100 board, the second X8 channel is allocated to the Huawei Ascend 310P board, and the X4 channel is allocated to the Huawei Ascend 310B board for use in EP mode.

2. A heterogeneous high-performance general-purpose deep learning edge computing platform according to claim 1, characterized in that: The front panel includes an indicator light, a USB interface, a VGA interface, a reset switch and a PS2 interface related to the mainboard.

3. A heterogeneous high-performance general-purpose deep learning edge computing platform according to claim 1, characterized in that: The rear panel includes audio interface, serial port, display interface and network interface related to the motherboard, as well as peripheral interfaces such as USB interface, HDMI interface, Gigabit network port and serial port in the RC mode of the Ascend 310B module.

4. A heterogeneous high-performance general-purpose deep learning edge computing platform according to claim 1, characterized in that: The external device data first obtains image data, audio data or other external data through the host CPU, and then performs relevant preprocessing according to application requirements. The data is then distributed to each accelerator card through the PCIE bus for hardware encoding and decoding and neural network calculations. The CPU's time-consuming calculations are completed in a short time within the NPU, and then returned to the CPU through the PCIE bus for subsequent data integration and output display.

5. The heterogeneous high-performance general-purpose deep learning edge computing platform according to claim 1, characterized in that: The logical software structure of the artificial intelligence computing system of the platform includes: the underlying hardware layer, the middle layer, the pooling layer and the business layer; the underlying hardware layer needs to adapt the interfaces of each intelligent computing module to the corresponding functional interfaces of the Feiteng D2000 host module according to the above-mentioned hardware platform architecture; in the middle layer, exclusive Docker containers for various types of computing modules are built, and the containers include computing module drivers, development and reasoning tool chains, and business applications, etc.; in the pooling layer, heterogeneous computing resources are managed in a unified manner, and a unified programming interface for reasoning applications and a high-performance model reasoning framework are built to facilitate the rapid calling and transplantation of business layer applications.

6. A heterogeneous high-performance general-purpose deep learning edge computing platform according to claim 5, characterized in that: The Docker container needs to perform environment deployment and software configuration from the physical machine layer and virtualized container layer respectively; the physical machine layer needs to complete the firmware installation of each intelligent computing module and the adaptation of the underlying driver; The specific method is as follows: first, select the driver package and firmware package corresponding to the CPU and operating system kernel version to complete the installation of the underlying driver and firmware; then, in the virtualization container layer, first create a dedicated Docker container belonging to the module. All subprocesses of each tool chain run in the container. In the container, all application environments are isolated from each other. The container communication method is used to interact with the software platform, so that the platform can call each development tool chain separately to complete the entire process from data upload to model deployment to program operation. The file system in the Docker container can be recreated or mounted on the host file system, and then the driver of the corresponding module is installed or mounted so that it can be successfully called in the Docker container; finally, the corresponding development tool chain is installed to run the application in the container.

7. A heterogeneous high-performance general-purpose deep learning edge computing platform according to claim 5, characterized in that: The pooling layer constructs a unified programming interface for heterogeneous platforms, builds a heterogeneous hardware adapter based on the computing interface of Huawei Ascend processors and Tianshu Zhixin processors, and uses functions such as hardware device management and model networking to open up the connection between the programming interface and the inference hardware; a unified programming interface is used at the upper layer to achieve access adaptation of multi-brand and multi-series AI computing chips, and then optimize the software stack according to the performance of each processor.

8. The heterogeneous high-performance general-purpose deep learning edge computing platform according to claim 5, characterized in that: The pooling layer also builds a multi-model high-performance synchronous reasoning framework, the specific method of which is: using the hardware codec of the AI ​​acceleration chip to implement hardware decoding of the input image, reducing the computational load of the CPU, and then unifying the preprocessing methods of multiple models; then writing hardware post-processing operators, transferring the post-processing part of the CPU to the NPU for calculation, and finally applying multi-threaded parallel pipeline reasoning technology in the entire processing flow.

Images