Interrupt virtualization method for PCIe (Peripheral Component Interconnect Express) equipment based on Descriptor

Abstract

The invention discloses a PCIe (Peripheral Component Interconnect Express) equipment interrupt virtualization method based on an interrupt Descriptor, which belongs to the field of virtualization and comprises the following steps: pre-storing an interrupt remapping table and the interrupt Descriptor into a system memory by utilizing virtual machine software; judging whether the corresponding interrupt needs to be virtualized or not according to the virtual mark in the PCIe equipment interrupt packet sent by the PCIe equipment; if the interrupt needs to be virtualized, acquiring an interrupt remapping table of the interrupt from the system memory according to the index information in the interrupt packet of the PCIe equipment; according to the obtained interrupt remapping table, the interrupt Descriptor of the interrupt is obtained from the system memory, and the CPU processor completes interrupt virtualization by reading information in the interrupt Descriptor. According to the method provided by the invention, interrupt virtualization is realized through hardware assistance, so that the expenditure of software virtualization can be reduced, and the performance of software interrupt virtualization is enhanced.

Description

Technical Field

[0001] The invention relates to the field of virtualization design, and in particular to a PCIe device interrupt virtualization method based on interrupt descriptor. Background Art

[0002] Interrupts are a mechanism used by the CPU processor to process requests from external devices. The use of interrupts can save CPU computing resources by not frequently accessing devices to confirm whether the devices have requests to process. In order to use interrupts in the virtual CPU of a virtual machine, interrupts must be virtualized. Interrupt virtualization can be achieved entirely by software or with hardware assistance. By virtualizing interrupts, the virtual machine running on the physical CPU can sense interrupt requests from external devices and process them.

[0003] Among the current ways to implement interrupt virtualization, software is usually used to inject interrupts. For example, in a Linux system, when a running virtual machine receives an interrupt, it triggers the virtual machine to exit to the operating system kernel mode, in which the mapping conversion from physical interrupts to virtual interrupts is completed, and this information is stored in the virtual machine. After the virtual machine is back online, the virtual interrupt can be sensed. This method is called interrupt injection. This interrupt virtualization method requires frequent switching between virtual machine and kernel mode, which is a huge overhead for software resources and also has a certain impact on software performance. Summary of the invention

[0004] The purpose of the present invention is to provide a PCIe device interrupt virtualization method based on interrupt descriptor, through the hardware-assisted interrupt virtualization process, the virtual interrupt is directly mapped to the interrupt descriptor in the memory space, the software can directly obtain the virtual interrupt information through the descriptor, saving software resources. The present invention is implemented by the following technical solutions.

[0005] The present invention provides a PCIe device interrupt virtualization method based on interrupt Descriptor, comprising: Using virtual machine software to pre-store the interrupt remapping table and interrupt descriptor into the system memory; Determine whether the corresponding interrupt needs to be virtualized according to the virtual flag in the PCIe device interrupt packet sent by the PCIe device; If the interrupt needs to be virtualized, obtaining an interrupt remapping table of the interrupt from the system memory according to the index information in the PCIe device interrupt packet; According to the obtained interrupt remapping table, the interrupt Descriptor of the interrupt is obtained from the system memory, and the CPU processor completes the interrupt virtualization by reading the information in the interrupt Descriptor.

[0006] In practical applications, the relationship between the interrupt remapping table and the interrupt descriptor of the virtual interrupt must first be specified through the virtual machine software, and then instructions are sent to the hardware to control the hardware to write the relationship into the system memory. Then, the virtual flag in the PCIe device interrupt packet sent by the PCIe device is used to determine whether the interrupt needs to be virtualized, and then the interrupt remapping table of the interrupt is obtained from the system memory through the index information in the interrupt packet, and then the interrupt descriptor of the interrupt is retrieved through the interrupt remapping table to assist in completing the interrupt virtualization. This method implements interrupt virtualization through hardware assistance, which can reduce the overhead of software virtualization and enhance the performance of software interrupt virtualization.

[0007] Optionally, the interrupt remapping table and the interrupt descriptor are pre-stored in the system memory by using the virtual machine software, specifically: the virtual machine software controls the CPU processor to store the interrupt remapping table and the interrupt descriptor in the system memory by sending instructions to the CPU processor; Among them, the interrupt remapping table is a page table used to record the location of the interrupt Descriptor in the system memory. The CPU processor can obtain the interrupt Descriptor from the system memory according to the location information recorded in the interrupt remapping table. The interrupt Descriptor is an interrupt descriptor, and the interrupt Descriptor records the relationship between the PCIe device interrupt and the CPU core.

[0008] Optionally, if the interrupt does not need to be virtualized, the PCIe device interrupt packet sent by the PCIe device is parsed through the interface component of the PCIe device, and the core number and vector number of the interrupt are obtained and sent to the virtual machine software. After receiving the interrupt, the virtual machine software exits from the virtual machine to the operating system kernel. The operating system kernel translates the vector number of the interrupt into a delicate and required vector number and writes it into the virtual machine space. The virtual machine running the software can perceive the interrupt when it comes back online.

[0009] Optionally, the interrupt remapping table also records a valid flag of the interrupt remapping table and a vector number corresponding to the virtual interrupt.

[0010] Optionally, each of the vector numbers corresponds to a vector field in an interrupt descriptor, which is used to determine a specific virtual interrupt. The vector field in the interrupt descriptor has 256 bits in total, and can support a maximum of 256 vector numbers of virtual interrupts, so there are 256 different virtual interrupts.

[0011] Optionally, obtaining an interrupt descriptor of the interrupt from the system memory according to the obtained interrupt remapping table, and the CPU processor completing interrupt virtualization by reading information in the interrupt descriptor, includes: The PCIe device interface component retrieves the interrupt descriptor from the system memory, and then sends an atomic write request to the consistency processing component for writing 1 to the corresponding position in the vector field of the interrupt descriptor according to the vector number of the virtual interrupt in the interrupt remapping table; after the consistency processing component completes the atomic write operation on the interrupt descriptor and writes the interrupt descriptor back to the system memory, the PCIe device interface component generates a notification interrupt to notify the CPU processor, and the CPU processor assists in completing the interrupt virtualization by reading the information in the interrupt descriptor. The hardware-assisted interrupt virtualization method proposed by the present invention specifically includes the following steps: S1, the virtual machine software controls the CPU processor to write the interrupt remapping table information and interrupt Descriptor into the system memory in advance; S2, the virtual machine software controls the PCIe device to send an interrupt packet, and indicates in the interrupt packet that the interrupt is a virtual interrupt; S3, after receiving the virtual interrupt, the PCIe device interface component of the CPU processor obtains the interrupt remapping table from the system memory according to the index information in the interrupt packet; S4, after the PCIe device interface component retrieves the interrupt remapping table, it retrieves the interrupt descriptor from the system memory according to the location information of the interrupt descriptor in the system memory in the table; S5, the PCIe device interface component sends an atomic write request to the consistency processing component to write 1 to the corresponding position in the vector field of the interrupt descriptor according to the vector number of the virtual interrupt in the interrupt remapping table; S6, the consistency processing component completes the atomic write operation on the interrupt descriptor; S7, after the consistency processing component completes the atomic write to the interrupt descriptor, the PCIe device interface component generates a notification to the CPU core corresponding to the interrupt notification according to the interrupt CPU core number and vector number in the interrupt descriptor, to assist in completing the interrupt virtualization.

[0012] Optionally, for virtual machine software running on the CPU processor, the PCIe device interface component sends a notification interrupt to the virtual machine where the software is running according to the target core number that generates the notification interrupt. After receiving the notification interrupt, the virtual machine where the software is running obtains the descriptor of the notification interrupt, retrieves the descriptor from the system memory, and parses the corresponding interrupt according to the vector field of the descriptor.

[0013] Optionally, after a PCIe device interface component retrieves the interrupt descriptor, it changes the position to be modified, and then writes the interrupt descriptor back to the system memory. The PCIe device that processes the interrupt descriptor only performs the above operations on the interrupt descriptor until the interrupt descriptor is written back to the system memory. These three steps cannot be interrupted, that is, during this period, other PCIe interface components cannot modify this descriptor. Beneficial Effects

[0014] The present invention adopts a PCIe device interrupt virtualization method based on interrupt descriptor, which can enable the running virtual machine software to directly perceive the virtual interrupt without exiting to the kernel, and no longer requires the interrupt injection process of frequently exiting and going online. The interrupt virtualization time is greatly shortened through hardware assistance. Compared with ordinary interrupts, the hardware writes the virtual interrupt directly into the memory space that the software virtual machine can directly access, eliminating the process of ordinary interrupt software interrupt injection and saving the software virtualization overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Shown is a brief flowchart of hardware-assisted interrupt virtualization; Figure 2 The figure shows the comprehensive processing flow chart of interrupt virtualization software and hardware interaction. DETAILED DESCRIPTION

[0016] The following is a further description in conjunction with the accompanying drawings and specific embodiments. In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. Example

[0017] This embodiment introduces a descriptor-based PCIe device interrupt virtualization method, including: Using virtual machine software to pre-store the interrupt remapping table and interrupt descriptor into the system memory; Determine whether the corresponding interrupt needs to be virtualized according to the virtual flag in the PCIe device interrupt packet sent by the PCIe device; If the interrupt needs to be virtualized, obtaining an interrupt remapping table of the interrupt from the system memory according to the index information in the PCIe device interrupt packet; According to the obtained interrupt remapping table, the interrupt Descriptor of the interrupt is obtained from the system memory, and the CPU processor completes the interrupt virtualization by reading the information in the interrupt Descriptor.

[0018] In practical applications, the relationship between the interrupt remapping table and the interrupt descriptor of the virtual interrupt must first be specified through the virtual machine software, and then instructions are sent to the hardware to control the hardware to write the relationship into the system memory. Then, the virtual flag in the PCIe device interrupt packet sent by the PCIe device is used to determine whether the interrupt needs to be virtualized, and then the interrupt remapping table of the interrupt is obtained from the system memory through the index information in the interrupt packet, and then the interrupt descriptor of the interrupt is retrieved through the interrupt remapping table to assist in completing the interrupt virtualization. This method implements interrupt virtualization through hardware assistance, which can reduce the overhead of software virtualization and enhance the performance of software interrupt virtualization. Specifically, it includes the following contents: Table 1 shows the packet format of the PCIe device interrupt. The interrupt packet includes a flag indicating whether the interrupt needs to be virtualized. In Table 1, the virtual flag of the [4th] bit of the interrupt packet Address is used to indicate whether the interrupt needs to be virtualized. If the virtual flag is 1, it means that the interrupt needs to be virtualized. Then the hardware will obtain the interrupt remapping table of the interrupt from the system memory through the index information in the interrupt packet.

[0019] Table 1 Packet format of PCIe device interrupt

[0020] The interrupt remapping table is shown in Table 2. It is a page table used to record the location of the interrupt descriptor in the system memory. The hardware can obtain the interrupt descriptor information from the system memory through the interrupt remapping table. At the same time, the interrupt remapping table also records the interrupt remapping table valid flag and the vector number corresponding to the virtual interrupt, which corresponds to the vector field in the interrupt descriptor.

[0021] Table 2 PCIe device interrupt remapping table

[0022] As shown in Table 3, the information recorded in the interrupt descriptor includes the CPU core number and interrupt vector number that the virtual interrupt needs to notify, which need to be arranged in the memory in advance by the software as needed. At the same time, the vector field in the interrupt descriptor can support a maximum of 256 vector numbers of virtual interrupts. According to the 256-bit vector field, there can be 256 different virtual interrupts. The system software can determine the specific virtual interrupt based on the vector field.

[0023] Table 3 Descriptor recorded in memory content

[0024] This embodiment only considers the case where the interrupt is a virtual interrupt, such as Figure 1 As shown, the steps for hardware-assisted interrupt virtualization are as follows: S1, using the CPU processor to pre-write the interrupt remapping table information and interrupt Descriptor into the memory; S2, using the virtual machine software to control the PCIe device to send an interrupt packet, and indicating in the interrupt packet that the interrupt is a virtualized interrupt; S3, after receiving the virtual interrupt, the PCIe device interface component of the CPU processor obtains the interrupt remapping table from the system memory according to the index information in the interrupt packet; S4, after the PCIe device interface component retrieves the interrupt remapping table, it retrieves the interrupt descriptor from the system memory according to the location information of the interrupt descriptor in the system memory in the table; S5, the PCIe device interface component sends an atomic write request to the consistency processing component according to the vector number of the virtual interrupt in the interrupt remapping table to write 1 to the corresponding position in the vector field taken by the interrupt descriptor; S6, the consistency processing component completes the atomic write operation on the interrupt descriptor; S7, after the consistency processing component completes the atomic write to the interrupt descriptor, the PCIe device interface component generates a notification to the CPU core corresponding to the interrupt notification according to the interrupt CPU core number and vector number in the interrupt descriptor, to assist in completing the interrupt virtualization.

[0025] For the virtual machine software running on the CPU core, after receiving this notification interrupt, the virtual machine software directly reads back the interrupt descriptor from the memory and determines the type of virtual interrupt based on the vector field therein.

[0026] Among them, the atomic write operation of the interrupt descriptor is completed by the consistency processing component, including that after a PCIe device interface component retrieves the interrupt descriptor, until it writes the interrupt descriptor back to the memory, no other operations on the interrupt descriptor can be performed, and the consistency processing component needs to handle them uniformly. These three steps cannot be interrupted, that is, during this period, other PCIe interface components cannot modify this descriptor. Example

[0027] Based on Example 1, this example introduces a comprehensive processing process of software and hardware interaction to implement interrupt virtualization, such as Figure 2 As shown, Figure 2 The interrupt virtualization software and hardware interaction comprehensive processing flow shown requires the interaction between software and hardware, and specifically includes the following contents: S1, the software writes the information of the base address of the interrupt remapping table into the register of the hardware PCIe device interface, and writes the interrupt remapping table and descriptor into the memory. The base address is the starting address of the interrupt remapping table in the memory.

[0028] S2, the software controls the device to send an interrupt. If the device works on a virtual machine, the device can directly send a virtual interrupt, otherwise it sends a normal interrupt.

[0029] S3, after receiving the interrupt, the hardware will determine the virtual interrupt flag in the interrupt packet. If the flag is 1, it is a virtual interrupt, otherwise it is a normal interrupt.

[0030] S4, if the hardware determines that it is a normal interrupt, the hardware directly parses the target core number and vector number of the interrupt packet, and generates a normal interrupt notification target core. After receiving the normal interrupt, the software exits from the virtual machine to the operating system kernel to perform interrupt injection, and then the virtual machine is back online.

[0031] S5: If the hardware determines that it is a virtual interrupt, then: S5.1, the hardware retrieves the interrupt remapping table from the memory according to the table entry index information in the interrupt packet and the table entry base address provided by the software.

[0032] S5.2, after obtaining the remapping table, retrieve the descriptor from the memory according to the descriptor address in the table, and write the target vector number in the table into the corresponding position of the vector field of the descriptor.

[0033] S5.3, after the hardware modifies the descriptor, it rewrites the descriptor into the memory, and the atomicity of modifying the descriptor is guaranteed by the hardware.

[0034] S5.4, the hardware generates a notification interrupt and sends it to the corresponding core according to the notification interrupt core number in the descriptor. After receiving the notification interrupt, the software directly retrieves the descriptor from the memory and obtains the interrupt used by the virtual machine through the vector number recorded in the descriptor.

[0035] In simple terms, compared with ordinary interrupts, the hardware writes virtual interrupts directly into the memory space that the software virtual machine can directly access, eliminating the process of injecting ordinary interrupts into software interrupts. The execution speed of hardware is much faster than that of software exiting the virtual machine and then going online, so hardware-assisted interrupt virtualization can greatly optimize the virtualization performance of software.

[0036] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0037] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0038] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0039] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0040] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A PCIe device interrupt virtualization method based on interrupt descriptor, characterized in that: include: Using virtual machine software to pre-store the interrupt remapping table and interrupt descriptor into the system memory; Determine whether the corresponding interrupt needs to be virtualized according to the virtual flag in the PCIe device interrupt packet sent by the PCIe device; If the interrupt needs to be virtualized, obtaining an interrupt remapping table of the interrupt from the system memory according to the index information in the PCIe device interrupt packet; According to the obtained interrupt remapping table, the interrupt Descriptor of the interrupt is obtained from the system memory, and the CPU processor completes the interrupt virtualization by reading the information in the interrupt Descriptor.

2. The PCIe device interrupt virtualization method according to claim 1, characterized in that: The method of using the virtual machine software to pre-store the interrupt remapping table and the interrupt descriptor in the system memory is as follows: the virtual machine software controls the CPU processor to store the interrupt remapping table and the interrupt descriptor in the system memory by sending instructions to the CPU processor; Among them, the interrupt remapping table is a page table used to record the location of the interrupt Descriptor in the system memory. The CPU processor can obtain the interrupt Descriptor from the system memory according to the location information recorded in the interrupt remapping table. The interrupt Descriptor is an interrupt descriptor, and the interrupt Descriptor records the relationship between the PCIe device interrupt and the CPU core.

3. The PCIe device interrupt virtualization method according to claim 1, characterized in that: If the interrupt does not need to be virtualized, the PCIe device interrupt packet sent by the PCIe device is parsed through the interface component of the PCIe device, and the core number and vector number of the interrupt are obtained and sent to the virtual machine software. After receiving the interrupt, the virtual machine software exits from the virtual machine to the operating system kernel. The operating system kernel translates the vector number of the interrupt into a delicate and required vector number and writes it into the virtual machine space. The virtual machine running the software can perceive the interrupt when it comes back online.

4. The PCIe device interrupt virtualization method according to claim 2, characterized in that: The interrupt remapping table also records the interrupt remapping table valid flag and the vector number corresponding to the virtual interrupt.

5. The PCIe device interrupt virtualization method according to claim 4, characterized in that: Each of the vector numbers corresponds to a vector field in an interrupt descriptor, and is used to determine a specific virtual interrupt.

6. The PCIe device interrupt virtualization method according to claim 1, characterized in that: The obtaining of the interrupt descriptor of the interrupt from the system memory according to the obtained interrupt remapping table, and the CPU processor completing interrupt virtualization by reading information in the interrupt descriptor, including: The PCIe device interface component retrieves the interrupt descriptor from the system memory, and then sends an atomic write request to the consistency processing component according to the vector number of the virtual interrupt in the interrupt remapping table, which is used to write 1 to the corresponding position in the vector field of the interrupt descriptor; after the consistency processing component completes the atomic write operation on the interrupt descriptor and writes the interrupt descriptor back to the system memory, the PCIe device interface component generates a notification interrupt to notify the CPU processor, and the CPU processor assists in completing interrupt virtualization by reading the information in the interrupt descriptor.

7. The PCIe device interrupt virtualization method according to claim 6, characterized in that: The PCIe device interface component sends a notification interrupt to the virtual machine running the software according to the target core number of the generated notification interrupt. After receiving the notification interrupt, the virtual machine running the software obtains the descriptor of the notification interrupt, retrieves the descriptor from the system memory, and parses the corresponding interrupt according to the vector field of the descriptor.

8. The PCIe device interrupt virtualization method according to claim 6, characterized in that: When a PCIe device interface component retrieves the interrupt descriptor, it changes the position to be modified and then writes the interrupt descriptor back to the system memory. The PCIe device that processes the interrupt descriptor only performs the above operations on the interrupt descriptor until the interrupt descriptor is written back to the system memory.

Images