A Method and Device for Managing PCI Bus Address of a Multi-CPU Architecture Virtual Machine

Abstract

The present application discloses a method and device for managing PCI bus addresses of a multi-CPU architecture virtual machine, including: configuring a primary PCI bus address with consecutive first address bits as a PCI bridge device, where any one of the PCI bridge devices corresponds to a secondary PCI bus, so as to access corresponding virtual PCI devices through the secondary PCI bus under the PCI bridge device, and no secondary PCI bridge device is mounted under any secondary PCI bus; configuring a primary PCI bus address with consecutive second address bits as a virtual on-board PCI device, where the second address bits do not overlap with the first address bits. Embodiments of the present application allocate virtual devices to specified PCI address ranges according to different virtual device types, make full use of the PCI address ranges, avoid insufficient PCI device numbers, and improve the manageability of virtual devices and the scalability of the bus.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method and device for managing PCI bus addresses of virtual machines with a multi-CPU architecture. Background Art

[0002] In some scenarios where QEMU and KVM are used to implement virtual machines, the virtual machine management tool calls the management interface provided by LIBVIRT to implement various configurations of the virtual machine.

[0003] For the i440fx virtual machine type, it only supports the PCI bus. For the i1440fx type virtual machine, generally LIBVIRT will place the virtual hardware device of the PCI interface type on the PCI bus 0 of this type of virtual machine. Since at most 32 devices can be attached to the PCI bus 0, this results in no available device numbers when more PCI devices need to be inserted. Although this situation can be solved by hot-inserting a PCI bridge device, the bus number of the dynamically hot-inserted PCI bridge device will be different after the virtual machine restarts compared to before, which will cause problems for application programs that rely on the PCI bus number.

[0004] The Q35 virtual machine type applicable to the X86_64 architecture only supports the PCIE bus. For this type of virtual machine, the PCIE root bus does not support hot plugging. During the creation phase of the virtual machine, several PCIE Root Ports are pre-created and these PCIE Root Ports are mounted on the PCIE root bus. Then LIBVIRT mounts the devices of the virtual machine on these pre-created PCIE Root Ports and starts the virtual machine through QEMU. Since the number of PCIE Root Ports created during the virtual machine creation phase is limited and some have been occupied by the virtual machine devices in the startup phase, the virtual machine devices available for dynamic hot plugging are very limited, resulting in no available PCI device numbers when new virtual devices are inserted. Similar to the i1440fx, this problem can also be solved by inserting a PCIE-to-PCI bridge device on the PCIE RootPort, but similarly, there will be a problem that the bus number of the dynamically hot-inserted PCI bridge device is different before and after the virtual machine restarts.

[0005] In the current method, the virtual machine management component does not implement the allocation of the PCI bus address of the virtual machine, which will lead to problems such as waste of the PCI bus address space, insufficient PCI addresses for virtual devices, and abnormal changes in the PCI addresses of virtual devices. Summary of the Invention

[0006] The embodiments of the present application provide a method and device for managing the PCI bus address of a multi-CPU architecture virtual machine. According to different types of virtual devices, they are assigned to a specified PCI address range, making full use of the PCI address range, avoiding insufficient PCI device numbers, and improving the manageability of virtual devices and the scalability of the bus.

[0007] The embodiments of the present application provide a method for managing the PCI bus address of a multi-CPU architecture virtual machine, including:

[0008] Configuring the primary PCI bus address with consecutive first address bits as a PCI bridge device, where any PCI bridge device corresponds to a secondary PCI bus, so as to access the corresponding virtual PCI device through the secondary PCI bus under the PCI bridge device, and no secondary PCI bridge device is mounted under any secondary PCI bus;

[0009] Configuring the PCI bus address with consecutive second address bits as a virtual mounted on-board PCI device, where the second address bit does not overlap with the first address bit.

[0010] Optionally, it further includes setting all PCI functions of the primary PCI bus to 0; and

[0011] It further includes setting all PCI functions of the secondary PCI bus to 0.

[0012] Optionally, configuring the primary PCI bus address with consecutive 12 address bits as a PCI bridge device;

[0013] Configuring the primary PCI bus address with the subsequent consecutive 19 address bits as a virtual mounted on-board PCI device.

[0014] Optionally, accessing the corresponding virtual PCI device through the secondary PCI bus under the PCI bridge device includes:

[0015] Designating consecutive third address bits under any secondary PCI bus as accessing the same type of virtual PCI device.

[0016] Optionally, it further includes directly putting into use or designating other device extensions for the unplanned secondary PCI bus addresses.

[0017] Optionally, the virtual PCI devices accessed through the secondary PCI bus include virtual network card virtio-net devices, virtual network card virtio-blk devices, virtual network card virtio-scsi devices, virtual network card e1000 devices, virtual network card vfio devices, and virtual network card vfio-user devices.

[0018] An embodiment of the present application further provides a computer device, including a processor and a memory. A computer program is stored on the memory, and when the computer program is executed by the processor, the steps of the foregoing method are implemented.

[0019] According to different types of virtual devices, embodiments of the present application allocate them to a specified PCI address range, make full use of the PCI address range, avoid insufficient PCI device numbers, and improve the manageability of virtual devices and the scalability of the bus.

[0020] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically listed below. Description of the Drawings

[0021] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present application. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0022] Figure 1 It is an example of the i440fx PCI bus architecture based on X86_64 according to an embodiment of the present application;

[0023] Figure 2 It is an example of the AARCH64 or Q35 PCI bus architecture based on X86_64 according to an embodiment of the present application. Detailed Embodiments

[0024] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0025] A virtual machine refers to a complete computer system with the functions of a complete hardware system simulated by software and running in a completely isolated environment. Using the physical hardware and computing resources of the host, an independent virtual operating system runs as a user space process in the host operating system. Like a physical machine, a virtual machine also requires dedicated devices to provide functions for the system, such as processing power, memory, storage, network, or graphics.

[0026] LIBVIRT is a set of virtualization management components that provides a management channel for upper-layer virtualization management components to configure and manage virtual machines, and converts these configuration information into the corresponding configurations of the underlying QEMU to achieve the configuration management of virtual machines.

[0027] KVM is a software of a virtual machine monitor. Through KVM, it is possible to separate the resources of a physical server from the hardware and allocate them appropriately for use by virtual machines. KVM supports the implementation of hardware-enhanced virtual machines based on architectures such as X86_64 and AARCH64. QEMU will simulate a complete virtual hardware platform on which a guest operating system can run, and manage how to allocate resources in the host and provide them to the guest. QEMU provides various IO devices such as storage, network, or graphics for virtual machines, as well as a PCI bus that connects these IO devices to virtual machines.

[0028] PCI is a local bus technology that connects computer peripherals to a computer system. PCI technology mainly includes PCI and PCIE. They are both used to connect devices to a computer. The difference is that they use different technologies for communication between devices on the bus.

[0029] The embodiments of the present application provide a method for managing PCI bus addresses of virtual machines with a multi-CPU architecture, including:

[0030] Configure the primary PCI bus addresses with consecutive first address bits as PCI bridge devices, where any one of the PCI bridge devices corresponds to a secondary PCI bus, so as to access the corresponding virtual PCI devices through the secondary PCI buses under the PCI bridge devices, and no secondary PCI bridge devices are mounted under any secondary PCI bus;

[0031] Configure the PCI bus addresses with consecutive second address bits as virtual on-board PCI devices, where the second address bits do not overlap with the first address bits.

[0032] In some embodiments, it further includes setting all PCI functions of the primary PCI bus to 0; and it further includes setting all PCI functions of the secondary PCI bus to 0.

[0033] In some embodiments, configure the primary PCI bus addresses with consecutive 12 address bits as PCI bridge devices;

[0034] Configure the subsequent consecutive 19 address bits of the primary PCI bus addresses as virtual on-board PCI devices.

[0035] A specific application example is that a total of 32 devices can be connected under the PCI bus root bridge (primary PCI bus), and the PCI function numbers are all set to 0. One exemplary configuration is that the bus addresses from 00:01.0 to 00:0c.0 are set as PCI bridge devices, with a total of 12 PCI bridges, and the 19 PCI addresses from 00:0d.0 to 00:1f.0 are used to virtually mount on-board PCI devices.

[0036] In the embodiments of the present application, according to different types of virtual devices, they are assigned to specified PCI address ranges, making full use of the PCI address ranges, avoiding insufficient PCI device numbers, and improving the manageability of virtual devices and the scalability of the bus.

[0037] In some embodiments, accessing corresponding virtual PCI devices through the secondary PCI bus under the PCI bridge device includes:

[0038] Designate consecutive third address bits under any secondary PCI bus to access the same type of virtual PCI devices.

[0039] In some embodiments, the virtual PCI devices accessed through the secondary PCI bus include the same type of virtual PC devices such as virtual network card virtio-net devices, virtual network card virtio-blk devices, virtual network card virtio-scsi devices, virtual network card e1000 devices, virtual network card vfio devices, virtual network card vfio-user devices, etc., and the same type of virtual PCI devices support up to 31 at most.

[0040] In some embodiments, it also includes directly putting into use or designating other device expansions for unplanned secondary PCI bus addresses.

[0041] Such as Figure 1 As shown, an optional application example is that the bus addresses from 00:01.0 to 00:0c.0 of the PCI bus root bridge (primary PCI bus) are set as PCI bridge devices, with a total of 12 PCI bridges, and the 19 PCI addresses from 00:0d.0 to 00:1f.0 are used to virtually mount on-board PCI devices.

[0042] The PCI bridge device is used to mount devices that support dynamic hot plugging. Each PCI bridge device corresponds to a new PCI bus (secondary PCI bus). No other PCI bridge devices are mounted under the PCI bridge device, but virtual PCI devices are directly mounted, and the PCI function is specified as 0. 32 PCI devices can be connected under each PCI (secondary PCI bus) bridge, and the following definitions can be executed:

[0043] Define the PCI addresses from 01:00.0 to 01:1f.0 for the virtio-net device of the virtual network card.

[0044] Define the PCI addresses from 02:00.0 to 02:1f.0 for the virtio-blk device of the virtual network card.

[0045] Define the PCI addresses from 03:00.0 to 03:1f.0 for the virtio-scsi device of the virtual network card.

[0046] Define the PCI addresses from 04:00.0 to 04:1f.0 for the e1000 device of the virtual network card.

[0047] Define the PCI addresses from 05:00.0 to 05:1f.0 for the vfio device of the virtual network card.

[0048] Define the PCI addresses from 06:00.0 to 06:1f.0 for the vfio-user device of the virtual network card.

[0049] The PCI addresses not yet planned for use can be used directly or for future expansion.

[0050] For the virt (AARCH64) and Q35 (X86_64) virtual machine types that only support the PCIE bus, the bus planning example proposed in this application is as Figure 2 shown. There can be a total of 31 PCIE Root Port devices under the PCIE bus root bridge, and each PCI function is set to 0. One PCIE-to-PCI bridge device (primary PCI bus) is attached to each of the PCIE Root Port devices with bus addresses from 00:01.0 to 00:0c.0, for a total of 12 bridge devices, and other virtual on-board PCI devices are attached. That is, corresponding to the case of the PCIE Root Port, as Figure 2 shown, in this example, a PCIE-to-PCI bridge device is connected to a PCIE Root Port address to form a primary PCI bus, so that a secondary PCI bus can be attached under the primary PCI bus.

[0051] The PCI bridge device is used to attach devices that support dynamic hot plugging. Each PCI bridge device corresponds to a new PCI bus (secondary PCI bus). No other PCI bridge devices are attached under the PCI bridge device. Instead, virtual PCI devices are directly attached, and the PCI function is specified as 0. 31 PCI devices can be attached under each PCI bridge, and the following definitions can be made:

[0052] Define the PCI addresses from 02:01.0 to 02:1f.0 for the virtio-net device of the virtual network card.

[0053] Define the PCI addresses from 04:01.0 to 04:1f.0 for the virtio-blk device of the virtual network card.

[0054] Define the PCI addresses from 06:01.0 to 06:1f.0 for the virtio-scsi device of the virtual network card.

[0055] Define the PCI addresses from 08:01.0 to 08:1f.0 for the e1000 device of the virtual network card.

[0056] Define the PCI addresses from 0a:01.0 to 0a:1f.0 for the vfio device of the virtual network card.

[0057] Define the PCI addresses from 0c:01.0 to 0c:1f.0 for the vfio-user device of the virtual network card.

[0058] The PCI addresses that have not been planned for use can be used directly or for future expansion.

[0059] After the above PCI bus configuration and address allocation, the virtual PCI devices located on the PCI bridge support hot plugging. Different types of virtual PCI devices are located in different PCI address segments, which is convenient for the management of the virtual machine management component. The same type of virtual PCI devices are located on the same PCI bus. Thus, the same type of devices appear as continuous PCI addresses inside the virtual machine, optimizing the management of the devices inside the virtual machine and facilitating the virtual machine management component to collect the device information of the virtual machine and display the report information.

[0060] The embodiment of this application also provides a computer device, including a processor and a memory. A computer program is stored on the memory, and when the computer program is executed by the processor, the steps of the foregoing method are implemented.

[0061] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, the element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the element.

[0062] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0063] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server or network device, etc.) to execute the methods described in various embodiments of the present application.

[0064] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims. All of these are within the protection scope of the present application.

Claims

1. A method for managing PCI bus addresses of a multi-CPU architecture virtual machine, characterized in that Including: Configuring the primary PCI bus address of consecutive first address bits as a PCI bridge device, where any one of the PCI bridge devices corresponds to a secondary PCI bus, so as to access the corresponding virtual PCI device through the secondary PCI bus under the PCI bridge device, and no secondary PCI bridge device is mounted under any secondary PCI bus; Configuring the primary PCI bus address of consecutive second address bits as a virtual mounted on-board PCI device, where the second address bit does not overlap with the first address bit.

2. The method for managing the PCI bus address of a multi-CPU architecture virtual machine according to claim 1, characterized in that, It further includes setting all the PCI functions of the primary PCI bus to 0; and It further includes setting all the PCI functions of the secondary PCI bus to 0.

3. The method for managing the PCI bus address of a multi-CPU architecture virtual machine according to claim 1, characterized in that, Configuring the primary PCI bus address of consecutive 12 address bits as a PCI bridge device; Configuring the primary PCI bus address of the subsequent consecutive 19 address bits as a virtual mounted on-board PCI device.

4. The method for managing the PCI bus address of a multi-CPU architecture virtual machine according to claim 1, wherein Accessing the corresponding virtual PCI device through the secondary PCI bus under the PCI bridge device includes: Designating consecutive third address bits under any secondary PCI bus as accessing the same type of virtual PCI device.

5. The method for managing the PCI bus address of a multi-CPU architecture virtual machine according to claim 4, characterized in that, It further includes directly putting into use or designating other device expansions for the unplanned secondary PCI bus addresses.

6. The method for managing the PCI bus address of a multi-CPU architecture virtual machine according to claim 4, characterized in that, The virtual PCI devices accessed through the secondary PCI bus include virtual network card virtio-net devices, virtual network card virtio-blk devices, virtual network card virtio-scsi devices, virtual network card e1000 devices, virtual network card vfio devices, and virtual network card vfio-user devices.

7. A computer device, characterized in that, Including a processor and a memory, where a computer program is stored on the memory, and when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images