FPGA virtualization implementation method for heterogeneous computing environment

Abstract

The application provides an FPGA virtualization implementation method for a heterogeneous computing environment, comprising: FPGA virtualization hardware architecture design: FPGA logic resources are divided into a static area and a dynamic area, the static area is an unconfigurable area, and the dynamic area is a plurality of dynamically reconfigurable vFPGA areas; host CPU end software design: application deployment, FPGA resource management are performed through a user application program API, and communication with the vFPGA area is performed; scheduling of the application is realized in space and time through a runtime manager; the plurality of dynamically reconfigurable vFPGA areas are instantiated through a driver program, required data structures are set for the vFPGA areas, and virtual memory mapping is created for an application program, so that communication with the FPGA is realized through a PCIe bus. The application improves the development efficiency, resource utilization and deployment flexibility of the FPGA heterogeneous computing platform as a whole.

Description

Technical Field

[0001] This invention relates to the field of embedded computing technology, and more specifically to an FPGA virtualization implementation method for heterogeneous computing environments. Background Technology

[0002] Currently, airborne information processing systems are facing an exponential explosion in data volume, and the required computing power is also continuously increasing. Current homogeneous multi-core general-purpose processing systems are no longer able to handle tasks such as large-scale information encoding / decoding, convolution, fast Fourier transform, and artificial intelligence recognition. FPGA devices, with their hardware flexibility, high throughput, and low power consumption, are increasingly being used in numerous computationally intensive tasks as coprocessors to achieve large-scale parallel acceleration processing. They also form heterogeneous computing systems with CPUs, further enriching and supplementing airborne distributed architectures.

[0003] Partially reconfigurable FPGA technology divides the FPGA internally into multiple regions, allowing for individual configuration of these regions during runtime. This enables online logic updates without affecting the functionality of other regions, allowing FPGAs to dynamically switch between multiple tasks in both time and space. However, current partially reconfigurable technologies only target single FPGA devices. For large-scale computing tasks, a single device cannot meet the throughput and real-time requirements. Airborne information processing systems typically break down tasks into multiple subtasks and distribute them to different computing nodes. Therefore, to expand computing power, multiple FPGA devices need to be cascaded to form an FPGA resource pool, and virtualization technology is used to abstract the logic resources, achieving unified resource management and scheduling. In terms of development, traditional hardware-based FPGA development requires application developers to master domain-specific low-level hardware knowledge, making it difficult to adapt to software instruction programming in heterogeneous computing systems, posing a significant challenge to application development. To meet the practical application needs of rapid equipment iteration and upgrades, a unified hardware and software development framework must be provided for application developers to decouple application development from hardware and utilize virtual memory resource management mechanisms to achieve efficient sharing of FPGA resources across multiple users and applications. Summary of the Invention

[0004] In view of this, the embodiments of this application provide an FPGA virtualization implementation method for heterogeneous computing environments. The present invention takes FPGA partial reconfiguration technology as the core, which can flexibly realize the spatiotemporal multiplexing of user logic, support the concurrent operation and dynamic switching of multiple applications, and improve the overall development efficiency, resource utilization and deployment flexibility of FPGA heterogeneous computing platform.

[0005] This application provides the following technical solution: a method for implementing FPGA virtualization in heterogeneous computing environments, comprising:

[0006] FPGA virtualization hardware architecture design: The FPGA logic resources are divided into static regions and dynamic regions. The static regions are non-configurable regions, and the dynamic regions are multiple dynamically reconfigurable vFPGA regions.

[0007] Host CPU-side software design: Application deployment, FPGA resource management, and communication with vFPGA regions are performed through user application APIs; application scheduling is implemented in space and time through the runtime manager; multiple dynamically reconfigurable vFPGA regions are instantiated through drivers, the required data structures are set for each vFPGA region, and virtual memory mappings are created for the application to communicate with the FPGA through the PCIe bus.

[0008] According to one embodiment of this application, a single vFPGA region comprises two parts: user logic and a dynamic packager.

[0009] According to one embodiment of this application, the user logic is a synthesized and system-verified bitstream, and the user can develop applications using languages ​​including HLS, Verilog, VHDL, and OpenCL; the dynamic packager provides a standard interface for the user logic, enabling the application to run across vFPGAs.

[0010] According to one embodiment of this application, the FPGA virtualization hardware architecture includes the following functional modules:

[0011] The XDMAPCIe core is used to implement DMA data transfer between the FPGA and the host CPU.

[0012] Dynamic Region Configuration Controller and Dynamic Region Isolator: The Dynamic Region Configuration Controller is used to directly write configuration data into the vFPGA region using the PCIe bus and the internal configuration access port ICAP to achieve partial reconfiguration; the Dynamic Region Isolator is used to ensure that the logic signals of the vFPGA region are decoupled from those of the static region during the partial reconfiguration phase.

[0013] The storage stack is used to support the XDMA PCIe core to realize data transfer between the CPU and FPGA, as well as local DDR resource sharing across vFPGAs;

[0014] The network protocol stack is used to interconnect FPGA computing nodes using the RDMA communication method based on the ROCEv2 network protocol, realize cross-FPGA data transmission, and form an FPGA resource pool.

[0015] The dynamic wrapper is used to provide a unified external interface for user logic and to realize virtual-to-physical address translation through the MMU unit with built-in TLB.

[0016] According to one embodiment of this application, all data communication in the vFPGA region is converted to physical addresses through address translation. Local storage requests are sent directly to the FPGA, while non-local requests are handled by the network protocol stack module.

[0017] According to one embodiment of this application, in the host CPU-side software design process, the driver is implemented in a modular manner, specifically including a device initialization module, a data read / write and control module, an interrupt handling module, and a device open, release, and unload module.

[0018] This invention proposes an FPGA virtualization framework for heterogeneous computing environments. At the hardware level, it is based on FPGA partially reconfigurable technology. Through a network protocol stack deployed on the FPGA, multiple FPGA devices are interconnected to form an FPGA resource pool, realizing high-speed interconnection and computing power fusion between multiple reconfigurable regions of multiple FPGAs. At the software level, the virtualization framework is used to abstract, allocate, schedule and manage all FPGA resources, providing a unified resource access interface for applications and decoupling the FPGA deployment of applications from the underlying hardware.

[0019] This invention abstracts the specific FPGA architecture and hardware resources, decoupling application development from hardware logic. With FPGA partial reconfiguration technology as the core, it provides a unified execution environment for user applications, enabling flexible spatiotemporal reuse of user logic resources, supporting concurrent operation and dynamic switching of multiple applications, and improving the overall development efficiency, resource utilization and deployment flexibility of the FPGA heterogeneous computing platform. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is an example of an FPGA virtualization hardware architecture.

[0022] Figure 2 This is a schematic diagram of the FPGA virtualization framework operation according to an embodiment of the present invention. Detailed Implementation

[0023] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments, providing a clear and complete description of the technical solutions of the present invention. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0025] like Figure 1 As shown, this embodiment of the invention provides an FPGA virtualization implementation method for heterogeneous computing environments, including:

[0026] FPGA virtualization hardware architecture design: FPGA logic resources are divided into static regions and dynamic regions. The static region is a non-configurable region, and the dynamic region is a plurality of dynamically reconfigurable vFPGA regions. A single vFPGA region contains two parts: user logic and dynamic packager.

[0027] Host CPU-side software design: Application deployment, FPGA resource management, and communication with vFPGA regions are performed through user application APIs; application scheduling is implemented in space and time through the runtime manager; multiple dynamically reconfigurable vFPGA regions are instantiated through drivers, the required data structures are set for each vFPGA region, and virtual memory mappings are created for the application to communicate with the FPGA through the PCIe bus.

[0028] This invention discloses an FPGA virtualization framework and implementation method for heterogeneous computing environments, comprising an FPGA virtualization hardware framework and software design, providing a unified execution environment for user applications. By virtually abstracting FPGA resources, it hides the underlying hardware details from application developers and provides corresponding interfaces to access virtual FPGA resources, thus decoupling application development from hardware logic. With FPGA partial reconfiguration technology at its core, it flexibly achieves spatiotemporal multiplexing of user logic. Simultaneously, through resource management and data isolation technologies, it enables efficient sharing and management of FPGA resources across multiple users and applications, comprehensively improving the development efficiency, resource utilization, and deployment flexibility of the FPGA heterogeneous computing platform.

[0029] Specifically, in terms of hardware design:

[0030] The hardware design divides FPGA logic resources into static and dynamic regions. The static region, as a non-configurable area, provides the necessary infrastructure for partially reconfigurable functions. This includes configuration controllers for both static and dynamic regions, dynamic configuration ports (ICAPs), dynamic region isolators, the logic required for the dynamic region (configuration ports, logic signal isolation, etc.), and the XDMA PCIe core responsible for communication with the CPU, network protocol stack, memory stack, and FPGA on-chip buses (control bus, data bus, and Smart Connect bus interconnect module). The dynamic region is further divided into a series of vFPGAs, essentially multiple dynamically reconfigurable FPGA regions. Relying on partially reconfigurable FPGA technology, this allows multiple applications to run concurrently and switch dynamically. Furthermore, a single vFPGA is divided into user logic and dynamic packages. The user logic is a synthesized and system-verified bitstream with high programming flexibility: users can develop applications using languages ​​such as HLS, Verilog, VHDL, and OpenCL. The dynamic package provides a standard interface for the user logic, enabling applications to run across vFPGAs. Figure 1 As shown, the FPGA virtualization hardware architecture includes the following core modules:

[0031] (1) XDMAPCIe core: responsible for implementing DMA data transfer between FPGA and host CPU, enabling users to perform direct memory transfer from CPU to FPGA (H2C) and FPGA to CPU (C2H).

[0032] (2) Dynamic Region Configuration Controller and Dynamic Region Isolator: The Dynamic Region Configuration Controller is responsible for directly writing configuration data into the vFPGA using the PCIe channel and the Internal Configuration Access Port (ICAP) to achieve partial reconfiguration. Dynamic region isolation ensures that the vFPGA is decoupled from the static region logic signals during the partial reconfiguration phase.

[0033] (3) Storage stack: Supports XDMA to realize data transfer between CPU and FPGA, as well as local DDR resource sharing across vFPGA.

[0034] (4) Network protocol stack: Using RDMA communication technology based on the ROCEv2 network protocol, FPGA computing nodes are interconnected to realize zero-copy data transmission across FPGAs and form an FPGA resource pool.

[0035] (5) Dynamic Packer: On the one hand, the dynamic packer provides a unified external interface for user logic. On the other hand, it realizes virtual-to-physical address translation through the MMU unit of the built-in TLB. The vFPGA's TLB contains a 4KiB page and a 2MiB page. All accesses of user logic to FPGA and host memory use the same unified TLB interface and are performed in parallel through different paths. It also allows users to determine the size of the TLB according to their own applications.

[0036] In terms of host CPU-side software design:

[0037] Software design mainly consists of three layers: user application API, runtime manager, and driver.

[0038] (1) User Application API: User applications are programmed through a set of provided APIs for deployment, management and communication with vFPGA, supporting process, thread and task-level scheduling execution.

[0039] (2) Runtime Manager: Implements application scheduling in terms of space and time.

[0040] (3) Driver: Instantiate vFPGAs, set the required data structures for them, and then communicate with vFPGAs and create virtual memory mappings for user application code. The driver specifically includes modules such as device initialization, data reading and writing and control, interrupt handling, device opening, release and unloading.

[0041] like Figure 2 As shown in the figure, an FPGA virtualization framework for heterogeneous computing environments is provided by an embodiment of the present invention. The specific implementation of the architecture is as follows:

[0042] On the CPU side, users develop applications based on the provided API library, create task queues, and the runtime manager allocates tasks to vFPGAs using a scheduling algorithm, deploying user logic to the corresponding vFPGAs. The driver part sets up data structures for the set of vFPGAs to be supported and creates memory mappings for the application, subsequently communicating with the FPGA via the PCIe bus. In this invention, all data communication with the vFPGA is converted to physical addresses through address translation. Local storage requests are sent directly to the FPGA, while non-local requests are handled by the network stack module. On the FPGA side, during the partial reconfiguration phase, the user logic bitstream is transmitted to the configuration port through the dynamic reconfiguration path in the static region to complete the reconfiguration.

[0043] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for implementing FPGA virtualization in heterogeneous computing environments, characterized in that, include: FPGA virtualization hardware architecture design: The FPGA logic resources are divided into static regions and dynamic regions. The static regions are non-configurable regions, and the dynamic regions are multiple dynamically reconfigurable vFPGA regions. Host CPU-side software design: Application deployment, FPGA resource management, and communication with vFPGA regions are performed through user application APIs; application scheduling is implemented in space and time through the runtime manager; multiple dynamically reconfigurable vFPGA regions are instantiated through drivers, the required data structures are set for each vFPGA region, and virtual memory mappings are created for the application to communicate with the FPGA through the PCIe bus; A single vFPGA region comprises two parts: user logic and dynamic package. The user logic is a synthesized and system-verified bitstream, and users can develop applications using languages ​​including HLS, Verilog, VHDL, and OpenCL; the dynamic wrapper provides a standard interface for the user logic, enabling applications to run across vFPGAs; The FPGA virtualization hardware architecture includes the following functional modules: The XDMA PCIe core is used to implement DMA data transfer between the FPGA and the host CPU. Dynamic Region Configuration Controller and Dynamic Region Isolator: The Dynamic Region Configuration Controller is used to directly write configuration data into the vFPGA region using the PCIe bus and the internal configuration access port ICAP to achieve partial reconfiguration; the Dynamic Region Isolator is used to ensure that the logic signals of the vFPGA region are decoupled from those of the static region during the partial reconfiguration phase. The storage stack is used to support the XDMA PCIe core to realize data transfer between the CPU and FPGA, as well as local DDR resource sharing across vFPGAs; The network protocol stack is used to interconnect FPGA computing nodes using the RDMA communication method based on the ROCEv2 network protocol, realize cross-FPGA data transmission, and form an FPGA resource pool. The dynamic wrapper is used to provide a unified external interface for user logic and to realize virtual-to-physical address translation through the MMU unit with built-in TLB.

2. The FPGA virtualization implementation method for heterogeneous computing environments according to claim 1, characterized in that, All data communication in the vFPGA area is converted to physical addresses through address translation. Local storage requests are sent directly to the FPGA, while non-local requests are handled by the network protocol stack module.

3. The FPGA virtualization implementation method for heterogeneous computing environments according to claim 1, characterized in that, In the host CPU software design process, the driver is implemented in a modular manner, specifically including a device initialization module, a data read / write and control module, an interrupt handling module, and a device opening, releasing, and unloading module.

Images