A method and system for performing dynamic partial reconfigurable configuration of FPGA in a virtual machine, a storage medium, and an electronic device

Abstract

The present invention discloses a method for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine, comprising the following steps: processing an exception generated by a privileged instruction for configuring an FPGA of a virtual machine in a virtual machine monitor; constructing a virtual FPGA configuration interface device node in a virtual machine device list; using a Linux FPGA subsystem software code in a virtual machine operating system kernel; using an FPGA configuration driver code in a virtual machine operating system kernel; and configuring an FPGA in the virtual machine through an interface exposed by the Linux FPGA subsystem.

Description

Technical Field

[0001] The present invention relates to the fields of computer system structure, system virtualization and heterogeneous processing system, and in particular to a method and system for performing dynamic partial reconfigurable configuration of FPGA in a virtual machine. Background Art

[0002] As Moore's Law slows down and new applications such as artificial intelligence on the cloud have a surging demand for computing power, general-purpose CPUs are increasingly unable to meet the needs of these applications. In order to address computing bottlenecks and reduce CPU burdens, more and more accelerators are being deployed in the cloud, such as graphics processing units (GPUs), field programmable gate arrays (FPGAs), tensor processing units (TPUs), and other types of hardware accelerators are emerging in commercial clouds.

[0003] Due to its hardware programmability and high computing energy efficiency, FPGAs have been widely deployed in the cloud for application acceleration. Cloud vendors such as Amazon, Microsoft, and Alibaba deploy FPGAs in the cloud and data centers to accelerate applications such as machine learning, graph computing, video image processing, blockchain, and genetic analysis.

[0004] The deployment of FPGA accelerators relies on dynamic partial reconfiguration technology to reconfigure the FPGA. Dynamic partial reconfiguration technology divides the FPGA into a static area (SHELL) and a dynamic area (ROLE) where the FPGA accelerator is deployed, ensuring that when the dynamic area of ​​the user accelerator is reconfigured, the static area and the dynamic areas of other accelerators will remain unchanged and unaffected.

[0005] In order to provide software and driver support for FPGA reconfiguration, the Linux kernel includes an abstract framework called the FPGA subsystem, which establishes a unified way to configure FPGAs, so that different FPGAs have a unified configuration interface. The FPGA subsystem separates the user space layer from the vendor-specific hardware layer by providing a common method and interface for FPGA reconfiguration.

[0006] For dynamic partial reconfiguration, the FPGA subsystem has two core modules: FPGA Manager and FPGA Bridge. Their locations in the Linux kernel are as follows: Figure 1 As shown in the figure. The unified configuration interface exposed by FPGA Manger can be used to program FPGAs from different FPGA manufacturers, which hides the underlying driver registered by a specific manufacturer's FPGA. For example, in Figure 1In the Xilinx ZYNQ device, the Processor Configuration Access Port (PCAP) low-level driver is registered with the FPGA Manager; the FPGA Bridge handles the decoupling of each FPGA partially reconfigurable area to prevent signal interference when the part is reconfigurable. Similar to the FPGA Manager, the FPGA Bridge exposes the vendor-independent decoupling method in user space and hides the low-level driver registered by a specific vendor. For example, in Figure 1 The Partial Reconfiguration (PR) decoupler driver supports signal decoupling during partial reconfiguration.

[0007] However, the existing Linux FPGA subsystem is not designed for virtual machines, and neither the FPGA Manager nor the FPGA Bridge can be seen in Linux virtual machines. Therefore, it is not feasible for tenants to dynamically configure their corresponding FPGA regions in virtual machines to deploy hardware accelerators.

[0008] However, reconfiguring hardware accelerators in a virtual machine is an important requirement for three main reasons:

[0009] 1) Deploying and running FPGA accelerators on FPGAs requires cloud providers to provide a control plane for configuring hardware accelerators for virtual machines;

[0010] 2) Providing the function of configuring accelerators in virtual machines can reduce the large startup overhead of virtual machines. If the hardware accelerator cannot be configured in the virtual machine running on the host, it needs to be configured in advance before the virtual machine is started. Then reconfiguring the hardware accelerator requires restarting the virtual machine. Therefore, configuring the FPGA accelerator in the virtual machine avoids restarting the virtual machine;

[0011] 3) In the FPGA development scenario, the iterative development and testing of the accelerator leads to inevitable frequent configuration. Therefore, it is very meaningful to provide an efficient method for configuring the FPGA accelerator in a virtual machine in this scenario.

[0012] Currently, it is not supported to partially configure the FPGA as multiple ROLEs in a virtual machine. Amazon AWS F1 instances define Amazon FPGA Images (AFI) based on the vendor bitstream (bitstream, including descriptions of FPGA hardware logic, routing, and register and on-chip memory initial values, which can configure the FPGA). After completing the FPGA design, tenants can use the provided software interface to deploy their registered AFI on the F1 instance. AWS F1 instances expose physical functions (PF) for management in the static logic of the FPGA, so that tenants can configure AFI through Linux Shell commands provided in user mode to reconfigure the hardware accelerator. However, Amazon's PF-based method for configuring FPGA hardware accelerators is not universal, especially in situations where the FPGA and host are not interconnected by PCIe, such as SoC FPGAs. In addition, this method only supports configuring the FPGA as a single ROLE in a virtual machine, and does not support configuring the FPGA as multiple ROLEs. Summary of the invention

[0013] In view of the above technical problems existing in the prior art, an embodiment of the present invention proposes a method for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine, comprising:

[0014] Step 1, handling the exception generated by the FPGA configuration privileged instruction in the virtual machine monitor;

[0015] Step 2: construct a virtual FPGA configuration interface device node in the virtual machine device list;

[0016] Step 3, using the Linux FPGA subsystem software code in the virtual machine operating system kernel;

[0017] Step 4, using the driver code for configuring the FPGA in the virtual machine operating system kernel; and

[0018] Step 5: Configure the FPGA in the virtual machine through the interface exposed by the Linux FPGA subsystem.

[0019] The embodiment of the present invention further provides a system for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine, comprising:

[0020] The first module is used to process the exceptions generated by the FPGA configuration privileged instructions in the virtual machine monitor;

[0021] The second module is used to construct a virtual FPGA configuration interface device node in the virtual machine device list;

[0022] The third module is used to use the Linux FPGA subsystem software code in the virtual machine operating system kernel;

[0023] A fourth module is used to use the driver code for configuring the FPGA in the virtual machine operating system kernel; and

[0024] The fifth module is used to configure the FPGA in the virtual machine through the interface exposed by the Linux FPGA subsystem.

[0025] An embodiment of the present invention further proposes an electronic device, comprising: a processor and a memory, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the above-mentioned method for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine is implemented.

[0026] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine is implemented.

[0027] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Figure 1 shows the ARM Linux kernel FPGA configuration software stack.

[0029] Figure 2 A flow chart of a method for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine according to an embodiment of the present invention is shown.

[0030] Figure 3 Shows a virtual PCAP device tree node built in the virtual machine.

[0031] Figure 4 Shows the virtual firmware device tree node built in the virtual machine.

[0032] Figure 5 The system architecture of the virtual machine HVC configuration method is shown.

[0033] Figure 6 The specific process of the virtual machine HVC configuration method is shown.

[0034] Figure 7 A module diagram of a system for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0035] The structural principle and working principle of the present invention are described in detail below in conjunction with the accompanying drawings:

[0036] When the inventor was conducting research on configuring FPGA accelerators in virtual machines, he found that there is currently no method to configure FPGA as multiple ROLEs in virtual machines. Users can configure FPGA through the abstract interface provided by the Linux FPGA subsystem in the physical machine, but the Linux FPGA subsystem is not visible in the virtual machine. Therefore, dynamic partial reconfiguration cannot be performed in the virtual machine and the FPGA hardware accelerator cannot be configured as multiple ROLEs. The inventor found that by exposing virtual device nodes and corresponding FPGA configuration drivers in the virtual machine, and using the virtual machine Linux FPGA subsystem software code, the same FPGA configuration interface as the physical machine can be exposed in the virtual machine. Secondly, by configuring the virtual machine-specific device driver properties, the FPGA configuration process of the virtual machine can enter the virtual machine monitor in the form of privileged instruction traps. Finally, in the virtual machine monitor, the FPGA can be configured normally by handling the exceptions generated by the FPGA configuration.

[0037] The present invention has the following key points:

[0038] (1) The same Linux FPGA subsystem abstract framework as the host is exposed in the virtual machine. Tenants can use the same interface as the physical machine to implement dynamic partial reconfiguration in the virtual machine to configure the FPGA.

[0039] (2) Through specific configuration, privileged instructions are initiated when configuring the FPGA in the virtual machine and intercepted in the virtual machine monitor to configure the FPGA.

[0040] Therefore, the method for performing dynamic partial reconfigurable configuration of FPGA in a virtual machine proposed in the present invention supports the configuration of FPGA hardware accelerators in virtual machines running on a host, for example, SoC FPGA platforms and PCIe plug-in card FPGA platforms, and has low overhead, thus avoiding the large virtual machine startup overhead caused by restarting the virtual machine to configure the FPGA accelerator.

[0041] The specific principle is introduced below using ARM architecture as an example.

[0042] See also Figure 2 In this embodiment, the present invention proposes a method for performing dynamic partial reconfiguration of an FPGA in a virtual machine, comprising: steps S1 to S5. Specifically:

[0043] Step S1, processing exceptions generated by privileged instructions for configuring a virtual machine FPGA in a virtual machine monitor, including but not limited to hypervisor calls (Hypervisor Call, HVC).

[0044] In order to implement the addition of FPGA configuration privileged instruction exception handling code in the virtual machine monitor, the method specifically includes: step S1.1, obtaining the FPGA accelerator bitstream buffer transmitted by the virtual machine through address translation from the client physical address to the host physical address in the virtual machine monitor; step 1.2, allocating a continuous buffer and copying the FPGA accelerator bitstream into it; step 1.3, using privileged instructions (including but not limited to SMC) to configure the FPGA. Step S2, constructing a virtual FPGA configuration interface device node in the virtual machine device list.

[0045] In order to realize the construction of a virtual FPGA configuration interface device node, such as PCAP and firmware, in the virtual machine device list, the device list refers to the device information when the computer system is started, such as the device information recorded in the ARM device tree; specifically, it includes: constructing a virtual PCAP device node in the virtual machine device tree (see Figure 3 ); build a virtual firmware device node in the virtual machine device tree and change its method to "hvc" (see Figure 4 ). Step S3, using the Linux FPGA subsystem software code in the virtual machine operating system kernel, including the Linux FPGAManager software code and the FPGABridge software code.

[0046] Step S4: using the driver code for configuring the FPGA in the virtual machine operating system kernel, including but not limited to the PCAP driver code and the firmware driver code.

[0047] The driver code for configuring the FPGA used in the virtual machine kernel specifically includes: using the PCAP driver code in the virtual machine kernel and using the firmware driver code in the virtual machine kernel.

[0048] Step S5, configuring the FPGA in the virtual machine through the interface exposed by the Linux FPGA subsystem.

[0049] In order to configure the FPGA through the interface exposed by the Linux FPGA subsystem in the virtual machine, it specifically includes: configuring part of the reconfigurable area through the interface exposed by the FPGAManager in the Linux FPGA subsystem in the virtual machine.

[0050] In combination with the above, the system framework for dynamically partially reconfiguring FPGA configuration in a virtual machine proposed by the present invention is as follows: Figure 5As shown, each tenant allocated with an FPGAPR area initiates privileged instructions (including but not limited to hypervisor calls (HVC)) to configure the FPGA through an FPGA configuration driver (including the Linux FPGAManager framework, including but not limited to PCAP drivers and firmware drivers), and implements a custom FPGA configuration privileged instruction processing function in the virtual machine monitor to accept virtual machine configuration requests, which ultimately performs actual FPGA configuration through specific instructions (including but not limited to ARM secure monitor calls (SMC)).

[0051] Reconfiguring the FPGA in a virtual machine relies on privileged instructions. The privilege level of ARM is also called the exception level (EL), and there are four privilege levels. EL0 is the user state, with the lowest privilege level, and applications usually run at this privilege level; EL1 is the kernel state, and the operating system usually runs at this privilege level; EL2 is used for virtualization scenarios, and the virtual machine monitor usually runs at this privilege level; EL3 is responsible for switching between the normal world and the secure world. In ARM's early kernel virtual machine (KVM) virtualization implementation, KVM was split into two parts: EL2 lowvisor and EL1 highvisor.

[0052] The specific implementation of reconfiguring FPGA in a virtual machine is as follows Figure 6 As shown in the figure, it includes the following key contents: (1) Using the Linux FPGAManager software framework in the virtual machine (corresponding to Figure 6 fpga-mgr software framework in the virtual machine). Provide tenants with the same user-mode sysfs file system configuration interface as the host machine in the virtual machine; (2) Build a virtual FPGA configuration interface device (including but not limited to PCAP and firmware) in the virtual machine device tree, see Figure 3 and Figure 4, especially in the firmware device node, the method attribute "smc" (i.e., use SMC privileged instructions) is changed to "hvc" (i.e., use HVC privileged instructions), and the driver code corresponding to the configuration hardware is used. Therefore, in the virtual machine, the FPGAManager system will register the virtual FPGA configuration interface hardware device (including but not limited to PCAP), so that the configuration operation initiated by the virtual machine user state can run normally until the HVC call is initiated in the firmware driver; (3) In order to handle the virtual machine HVC call for FPGA configuration, the corresponding HVC exception handling function is implemented in KVM. In the HVC exception handling function, the relevant configuration parameters passed by HVC (such as whether it is partially reconfigurable and the size of the configuration bitstream) are obtained, and the accelerator bitstream file passed by the virtual machine is obtained through address translation, and finally the actual FPGA configuration is performed by initiating an SMC call. The SMC instruction finally enters the ARM trusted firmware for execution, and finally the FPGA configuration is performed through inter-core communication with the ARM Platform Management Unit (Platform Management Unit, PMU).

[0053] In addition, see Figure 7 As shown, based on the same inventive concept, an embodiment of the present invention further proposes a system 100 for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine, comprising:

[0054] The first module 110 is used to process the exception generated by the virtual machine FPGA configuration privileged instruction in the virtual machine monitor.

[0055] The second module 120 is used to construct a virtual FPGA configuration interface device in the virtual machine device list;

[0056] The third module 130 is used to use the Linux FPGA subsystem software code in the virtual machine operating system kernel;

[0057] A fourth module 140 is used to use the driver code for configuring the FPGA in the virtual machine operating system kernel; and

[0058] The fifth module 150 is used to configure the FPGA in the virtual machine through the interface exposed by the Linux FPGA subsystem.

[0059] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the methods, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0060] Based on the same inventive concept, an embodiment of the present invention also proposes an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be run on the processor. When the computer program is executed by the processor, a method for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine of the above embodiment is implemented.

[0061] Based on the same inventive concept, an embodiment of the present invention further proposes a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, a method for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine according to the above embodiment is implemented.

[0062] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for performing dynamic partial reconfiguration of an FPGA in a virtual machine, characterized in that: include: Step 1, handling the exception generated by the FPGA configuration privileged instruction in the virtual machine monitor; Step 2, construct a virtual FPGA configuration interface device node in the virtual machine device list; Step 3, using the Linux FPGA subsystem software code in the virtual machine operating system kernel; Step 4, using the driver code for configuring the FPGA in the virtual machine operating system kernel; as well as Step 5, configure the FPGA in the virtual machine through the interface exposed by the Linux FPGA subsystem; The step 1 specifically includes: step 1.1, obtaining the FPGA accelerator bitstream buffer transmitted by the virtual machine through address translation from the client physical address to the host physical address in the virtual machine monitor; step 1.2, allocating a continuous buffer and copying the FPGA accelerator bitstream therein; and step 1.3, configuring the FPGA using privileged instructions; The step 5 specifically includes: configuring part of the reconfigurable area in the virtual machine through the interface exposed by the FPGA Manager in the Linux FPGA subsystem; Each tenant allocated with an FPGAPR area initiates privileged instructions to configure the FPGA through the FPGA configuration driver, and implements a custom FPGA configuration privileged instruction processing function in the virtual machine monitor to accept virtual machine configuration requests, which ultimately performs actual FPGA configuration through specific instructions.

2. The method for performing dynamic partial reconfiguration of FPGA in a virtual machine according to claim 1, characterized in that: The privileged instructions configured in step 1 include ARM hypervisor calls.

3. The method for performing dynamic partial reconfiguration of FPGA in a virtual machine according to claim 1, characterized in that: The device list in step 2 is the device information when the computer system is started.

4. The method for performing dynamic partial reconfiguration of FPGA in a virtual machine according to claim 1, characterized in that: The Linux FPGA subsystem used in the virtual machine operating system kernel in step 3 includes the Linux FPGA Manager and the FPGA Bridge.

5. The method for performing dynamic partial reconfigurable configuration of FPGA in a virtual machine according to claim 1, characterized in that: The step 4 uses the driver code for configuring the FPGA in the virtual machine operating system kernel, including the FPGA configuration interface driver code.

6. A system for performing dynamic partial reconfigurable configuration of an FPGA in a virtual machine, characterized in that: include: The first module is used to handle exceptions generated by FPGA configuration privileged instructions in the virtual machine monitor; The second module is used to construct a virtual FPGA configuration interface device node in the virtual machine device list; The third module is used to use the Linux FPGA subsystem software code in the virtual machine operating system kernel; The fourth module is used to use the driver code for configuring the FPGA in the virtual machine operating system kernel; as well as The fifth module is used to configure the FPGA in the virtual machine through the interface exposed by the Linux FPGA subsystem; The first module specifically includes: obtaining the FPGA accelerator bitstream buffer transmitted by the virtual machine through address translation from the client physical address to the host physical address in the virtual machine monitor; allocating a continuous buffer and copying the FPGA accelerator bitstream therein; and configuring the FPGA using privileged instructions; The fifth module specifically includes: configuring a portion of the reconfigurable area in the virtual machine through an interface exposed by the FPGA Manager in the Linux FPGA subsystem; Each tenant allocated with an FPGAPR area initiates privileged instructions to configure the FPGA through the FPGA configuration driver, and implements a custom FPGA configuration privileged instruction processing function in the virtual machine monitor to accept virtual machine configuration requests, which ultimately performs actual FPGA configuration through specific instructions.

7. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images