FPGA (Field Programmable Gate Array) virtualization hardware system stack design for cloud deep learning reasoning

Abstract

The invention discloses an FPGA (Field Programmable Gate Array) virtualization hardware system stack design for cloud deep learning reasoning, and relates to the technical field of artificial intelligence, the design comprises a distributed FPGA hardware auxiliary virtualization hardware architecture, a CPU (Central Processing Unit) server node for running a virtual machine container, a static compiler and a deep neural network DNN, wherein the deep neural network DNN is used for acquiring a user instruction, and the user instruction is compiled into an instruction packet through the static compiler; the FPGA server computing node is used for operating virtualization system service and an FPGA acceleration card, and the FPGA acceleration card comprises a virtualization multi-core hardware resource pool and four double-rate synchronous dynamic random access memories (DDRs); and the master control node is used for managing each node in the CPU server node and the FPGA server computing node through a control layer. According to the scheme, the technical problem that in the prior art, an FPGA virtualization scheme oriented to deep learning reasoning application cannot be expanded to a distributed multi-node computing cluster is solved.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a design of an FPGA virtualized hardware system stack oriented to cloud-based deep learning reasoning. Background technique [0002] We are in the era of rapid development of artificial intelligence, and deep learning is playing an increasingly important role in various fields. Among them, the inference task of deep neural network (DNN) occupies most of the deep learning task load of cloud data center. The use of traditional general-purpose processors (CPUs) in data centers can no longer meet the huge computing power requirements of deep learning. Therefore, dedicated hardware platforms, such as GPUs, FPGAs, and ASICs, are now commonly used to accelerate deep learning algorithms. Thanks to FPGA's good balance of programmability, performance, and power consumption, more and more cloud service providers, such as Amazon, Alibaba, and Baidu, have begun to de...

AI Technical Summary

Problems solved by technology

For the hardware layer, the existing multi-user multi-core virtualization accelerator [3] uses a fully connected method to make the memory bandwidth of each core equal to the slice, so as to achieve multi-user performance isolation; for the scheduling compilation layer, the existing scheduler and compiler [ 2,3,4] Traverse the performance of all resource allocation and scheduling methods, and select the optimal scheduling method; for the application layer, the mainstream virtualization framework uses an application program interface (API)-based method to remotely configure hardware through the scheduling compilation layer Layer virtualization resources, such as [5] the GPU virtualization framework uses the CUDA API on the client side to remotely use the CUDA Runtime Library of the computing node, which will generate about 105 API calls per second, resulting in up to 5 times Remote access latency overhead

Images