Smart network card virtual machine migration method based on libvirt and qemu

Abstract

This application discloses a method for migrating smart NIC virtual machines based on libvirt and qemu, including the following steps: On the source smart NIC virtual machine to be migrated, configuration is performed: on the smart NIC side, a dirty page recording module and a first Migrate-agent module are configured; on the qemu side, a vfio migration agent module and a qmp interface are configured; on the destination virtual machine receiving the migration, configuration is performed: on the smart NIC side, a second Migrate-agent module is configured; on the qemu side, a vfio migration agent module and a qmp interface are configured. After configuration, the virtual machine migration is performed. Using the method of this application embodiment, the migration of virtual machines with smart NIC devices is realized based on libvirt and qemu.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method for migrating a smart network interface card virtual machine based on libvirt and qemu. Background Technology

[0002] Smart network interface cards (NICs) are a new type of NIC that integrates networking and computing capabilities. They typically consist of a network interface controller and a multi-core CPU, and can be optionally equipped with an FPGA or GPU. Smart NICs can be viewed as programmable NICs that can perform networking, storage, and security functions. Furthermore, they can also enable virtualization, load balancing, and data path optimization.

[0003] Smart NICs integrate computing power, offloading network, storage, and security functions from the host to free up valuable resources such as CPU. Smart NICs offer a wide range of functions: offloading network protocols including VXLAN, NVGRE, Geneve, etc.; performing packet inspection, flow table processing, encryption, VXLAN overlay, and NVMe-oF; and acting as storage controllers, establishing connections with source storage and offloading I / O to the smart NIC without requiring host CPU involvement. For security, they can implement data stream encryption and decryption functions.

[0004] Libvirt is currently the most widely used tool and API for managing KVM virtual machines. Libvirtd is a daemon process that can be invoked locally by virsh or remotely by virsh. Libvirtd uses the interfaces provided by qemu to operate virtual machines.

[0005] QEMU is a software suite that emulates processors, enabling the simulation of entire computer systems. It is currently widely used in cloud computing, and when combined with KVM, it utilizes hardware-assisted virtualization technology to achieve high-performance virtual machines. Smart NIC technology is one such acceleration technology that offloads network and I / O loads to the NIC, and it is gradually gaining widespread adoption.

[0006] The common usage of smart network adapters currently involves creating multiple virtual virtualizers (VFs) and then passing these VFs directly to the virtual machine. However, QEMU cannot detect data changes in these VFs, such as DMA data transfers between the virtual machine and the smart network adapter, or the status of the smart network adapter. Therefore, existing libraries like libvirt and QEMU do not support migrating virtual machines in this manner. Summary of the Invention

[0007] This invention provides a method for migrating smart network interface virtual machines based on libvirt and qemu, which can be used to migrate virtual machines containing smart network interface devices.

[0008] This invention provides a method for migrating a smart network interface virtual machine based on libvirt and qemu, comprising the following steps:

[0009] Perform the following configuration on the source smart NIC virtual machine to be migrated:

[0010] Smart NIC side configuration:

[0011] The dirty page recording module is used to record the dirty page information of the DMA address of the virtual machine to be migrated;

[0012] The first Migrate-agent module is used to establish a socket listening port, receive network connections from the QEMU side, and during the migration process, transmit the dirty page information and the device information of the virtual machine to be migrated to QEMU. After the migration is completed, it receives a message to stop the device and notifies the smart network card.

[0013] QEMU side configuration:

[0014] The vfio migration agent module is used to establish communication and transmit data with the migration agent on the smart NIC side.

[0015] The qmp interface receives the IP and port information of the smart network interface card from libvirt;

[0016] Configure the destination virtual machine receiving the migration as follows:

[0017] Smart NIC side configuration:

[0018] The second Migrate-agent module establishes a socket listening port to receive network connections from the QEMU side. During the migration process, it receives device status and dirty page information transmitted from the QEMU side, and after the migration is completed, it receives the smart NIC start message from the QEMU side and starts the smart NIC.

[0019] QEMU side configuration:

[0020] The vfio migration agent module is used to establish communication and transmit data with the migration agent on the smart NIC side.

[0021] The qmp interface receives the IP and port information of the smart network interface card from libvirt;

[0022] After configuration, perform virtual machine migration.

[0023] Optionally, the vfio migration agent module is embedded in the qemu vfio migration framework, and the dirty page recording function is enabled before the virtual machine migration is performed.

[0024] Optionally, performing a virtual machine migration includes the following process:

[0025] The source-side smart NIC virtual machine receives the virsh migrate command, parses the parameters, and generates the destination-side virtual machine's XML file based on the device information of the source-side smart NIC virtual machine;

[0026] The source libvirtd passes the XML file to the destination libvirtd;

[0027] The destination libvirtd prepares the environment according to the XML file, generates the startup parameters for qemu, and pulls the qemu process. The qemu process establishes a listening port.

[0028] The source libvirtd calls the qemu qmp interface to migrate virtual machines.

[0029] Optionally, the virtual machine migration process may also include:

[0030] The source libvirt first sends the header information and migration configuration to the destination qemu;

[0031] The source QEMU marks all memory blocks as dirty and passes all memory blocks to the destination QEMU;

[0032] Traverse all memory blocks and perform memory content transfer.

[0033] Optionally, memory content transfer also includes:

[0034] Calculate the remaining amount during the transfer of memory contents;

[0035] If the remaining amount can be transferred in one go, pause the virtual machine and pass the contents of the remaining memory blocks and the device status information to the destination.

[0036] This application also proposes a terminal device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the aforementioned smart network card virtual machine migration method based on libvirt and qemu.

[0037] This application also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned smart network interface virtual machine migration method based on libvirt and qemu.

[0038] Using the method of this invention, the migration of virtual machines with smart network interface cards was achieved based on libvirt and qemu.

[0039] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0040] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0041] Figure 1 This is a basic flowchart of the smart network interface card virtual machine migration method according to an embodiment of this application. Detailed Implementation

[0042] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0043] This invention provides a method for migrating smart network interface virtual machines based on libvirt and qemu, such as... Figure 1 As shown, it includes the following steps:

[0044] In step S101, the following configuration is performed on the source-side smart NIC virtual machine to be migrated:

[0045] Source-side smart NIC configuration:

[0046] The dirty page recording module is used to record the dirty page information of the DMA address of the virtual machine to be migrated. Specifically, after the VF is passed through to the virtual machine, the smart NIC side has the DMA information of the NIC operation. In this example, the DMA information includes the GPA address and length of the dirty page.

[0047] The first Migrate-agent module is used to establish a socket listening port, receive network connections from the QEMU side, and during the migration process, transmit the dirty page information and the device information of the virtual machine to be migrated to QEMU. After the migration is completed, it receives the message to stop the device and notifies the smart NIC. After notifying the smart NIC, the smart NIC completes the final data packet transmission.

[0048] Source-side QEMU configuration:

[0049] A VFIO migration agent module is added to QEMU to establish communication and transmit data with the migration agent on the smart NIC side. This includes transmitting DMA dirty page information and device status; it also includes enabling and disabling dirty page recording. Furthermore, it provides various interfaces for QEMU device migration to complete the migration process.

[0050] The qmp interface receives the IP and port information of the smart network interface card from libvirt.

[0051] In step S102, the following configuration is performed on the destination virtual machine receiving the migration:

[0052] Destination-side smart NIC configuration:

[0053] The second Migrate-agent module establishes a socket listening port to receive network connections from the QEMU side. During the migration process, it receives device status and dirty page information transmitted from the QEMU side. After the migration is completed, it receives the smart NIC start message from the QEMU side and starts the smart NIC.

[0054] Destination QEMU side configuration:

[0055] The vfio migration agent module is used to establish communication and transmit data with the migration agent on the smart NIC side.

[0056] The qmp interface receives the IP and port information of the smart network interface card from libvirt;

[0057] In step S103, after the configuration is completed, the virtual machine migration is performed.

[0058] Specifically, you can use `virsh migrate` to add parameters such as `srcsmartnicip`, `srcsmartnicport`, `dsrsmartnicip`, and `dstsmartnicport`, which represent the IP address and port information of the source and destination smart network interfaces, respectively.

[0059] libvirtd sends the smart NIC's listening port information (IP address and port) to qemu, and both ends of the qemu will create a network connection with the smart NIC during the migration phase.

[0060] Then the virtual machine migration process is executed.

[0061] Using the method of this invention, the migration of virtual machines with smart network interface cards was achieved based on libvirt and qemu.

[0062] In some embodiments, the VFIO migration agent module is embedded in the QEMU VFIO migration framework to enable dirty page recording before performing virtual machine migration. Specifically, the VFIO migration agent module is embedded within the QEMU VFIO migration framework.

[0063] Before migration, enable the dirty page recording function. During memory migration, obtain information about dirty pages from the smart network card side, including GPA address and size, and migrate the memory to the destination.

[0064] In the final stage of the migration, the device status of the smart network interface card (NIC) is read and migrated to the destination, and the smart NIC is triggered to send data outward, sending the source's own network data to the network. Finally, the smart NIC is stopped.

[0065] The destination receives the dirty page memory, restores the dirty page information and device status on the smart network interface card side, and finally enables the smart network interface card.

[0066] In some embodiments, performing a virtual machine migration includes the following process:

[0067] The source smart NIC virtual machine receives the virsh migrate command, parses the parameters, checks the validity of the migration parameters, and verifies whether the current QEMU version supports these parameters. If the parameters are supported, it generates the destination virtual machine's XML file based on the device information of the source smart NIC virtual machine.

[0068] The source libvirtd passes the XML file to the destination libvirtd.

[0069] The destination libvirtd performs environment preparation based on the XML file, such as creating disk devices. Then it generates the QEMU startup parameters, pulls the QEMU process, and the QEMU process establishes a listening port.

[0070] The source libvirtd calls the qemu qmp interface to migrate virtual machines.

[0071] In some embodiments, the virtual machine migration process further includes:

[0072] The source libvirt first sends the header information and migration configuration to the destination qemu;

[0073] The source QEMU marks all memory blocks as dirty and passes them to the destination QEMU. Specifically, marking all memory blocks as dirty means that all memory blocks need to be passed, and the name and size of all memory blocks are passed to the destination QEMU.

[0074] Traverse all memory blocks and perform memory content transfer.

[0075] In some embodiments, transferring memory contents further includes:

[0076] Calculate the remaining amount during the transfer of memory contents;

[0077] If the remaining amount can be transferred in one go, pause the virtual machine and pass the contents of the remaining memory blocks and the device status information to the destination.

[0078] The destination QEMU receives the migration data from the memory and restores it as it is received. Once the migration is complete, the vCPU is started, and the destination virtual machine begins to run.

[0079] The source libvirtd stops the qemu process, clears virtual machine information in libvirtd, and completes the migration.

[0080] This application's solution can complete the migration of smart network interface virtual machines using the native libvirt+qemu approach, requiring minimal adaptation of the management plane and exhibiting good compatibility. The method for transmitting smart network interface information between libvirt and qemu can be customized to pass the necessary values ​​based on the specific situation. The dirty page recording module and migrate-agent module provided in this application are not limited to individual smart network interfaces and possess general applicability.

[0081] This application also proposes a terminal device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the aforementioned smart network card virtual machine migration method based on libvirt and qemu.

[0082] This application also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned smart network interface virtual machine migration method based on libvirt and qemu.

[0083] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0084] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0086] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for smart NIC virtual machine migration based on libvirt and qemu, characterized in that, Includes the following steps: Perform the following configuration on the source smart NIC virtual machine to be migrated: Smart NIC side configuration: The dirty page recording module is used to record the dirty page information of the DMA address of the virtual machine to be migrated; The first Migrate-agent module is used to establish a socket listening port, receive network connections from the QEMU side, and during the migration process, transmit the dirty page information and the device information of the virtual machine to be migrated to QEMU. After the migration is completed, it receives a message to stop the device and notifies the smart network card. QEMU side configuration: The vfio migration agent module is used to establish communication and transmit data with the migration agent on the smart NIC side. The qmp interface receives the IP and port information of the smart network interface card from libvirt; Configure the destination virtual machine receiving the migration as follows: Smart NIC side configuration: The second Migrate-agent module establishes a socket listening port to receive network connections from the QEMU side. During the migration process, it receives device status and dirty page information transmitted from the QEMU side, and after the migration is completed, it receives the smart NIC start message from the QEMU side and starts the smart NIC. QEMU side configuration: The vfio migration agent module is used to establish communication and transmit data with the migration agent on the smart NIC side. The qmp interface receives the IP and port information of the smart network interface card from libvirt; After configuration, perform virtual machine migration.

2. The method for migrating intelligent network interface virtual machines based on libvirt and qemu as described in claim 1, characterized in that, The vfio migration agent module is embedded in the qemu vfio migration framework and enables dirty page recording before performing virtual machine migration.

3. The method for migrating intelligent network interface virtual machines based on libvirt and qemu as described in claim 2, characterized in that, Performing a virtual machine migration includes the following process: The source-side smart NIC virtual machine receives the virsh migrate command, parses the parameters, and generates the destination-side virtual machine's XML file based on the device information of the source-side smart NIC virtual machine; The source libvirtd passes the XML file to the destination libvirtd; The destination libvirtd prepares the environment according to the XML file, generates the startup parameters for qemu, and pulls the qemu process. The qemu process establishes a listening port. The source libvirtd calls the qemu qmp interface to migrate virtual machines.

4. The method for migrating intelligent network interface virtual machines based on libvirt and qemu as described in claim 3, characterized in that, The virtual machine migration process also includes: The source libvirt first sends the header information and migration configuration to the destination qemu; The source QEMU marks all memory blocks as dirty and passes all memory blocks to the destination QEMU; Traverse all memory blocks and perform memory content transfer.

5. The method for migrating intelligent network interface virtual machines based on libvirt and qemu as described in claim 4, characterized in that, Transferring memory contents also includes: Calculate the remaining amount during the transfer of memory contents; If the remaining amount can be transferred in one go, pause the virtual machine and pass the contents of the remaining memory blocks and the device status information to the destination.

6. A terminal device, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the steps of the smart NIC virtual machine migration method based on libvirt and qemu as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the smart NIC virtual machine migration method based on libvirt and qemu as described in any one of claims 1 to 5.

Images