A virtual machine migration method, system, device, and storage medium

Abstract

This application provides a virtual machine migration method, system, device, and storage medium. A first acceleration device is connected to the source host, and a second acceleration device is connected to the destination host. During the virtual machine migration process, the first acceleration device can send the source address information of the last round of dirty pages in the source host to the second acceleration device, which then undertakes the migration of the last round of dirty pages. After sending the source address information, the first acceleration device becomes idle and can then undertake the migration of device state data. In this way, the migration of the last round of dirty pages undertaken by the second acceleration device and the migration of device state data undertaken by the first acceleration device can be performed in parallel, without waiting for the dirty page migration to complete before performing the device state data migration. This effectively hides part or all of the time spent on device state data migration, greatly shortening the virtual machine's pause time and thus effectively improving virtual machine migration efficiency.

Description

Technical Field

[0001] This application relates to the field of virtualization technology, and in particular to a virtual machine migration method, system, device and storage medium. Background Technology

[0002] In the cloud computing field, cloud servers are handling increasingly diverse customer needs, thus placing higher demands on their performance. I / O performance is one of the core performance indicators. To meet these ever-increasing I / O performance requirements, the industry is constantly developing new I / O devices for cloud servers. To support concurrent access performance at the application layer, the number of device queues corresponding to each virtual machine on a cloud server has increased to tens of thousands or even higher, resulting in a massive amount of data related to device status.

[0003] Currently, device state migration can only be performed serially after the final round of dirty page migration is complete, and this migration must be carried out while the virtual machine is paused. However, due to the sheer number of device states, this results in the virtual machine being paused for an excessively long time, significantly reducing the migration efficiency. Summary of the Invention

[0004] This application provides a virtual machine migration method, system, device, and storage medium in several aspects to improve the migration efficiency of virtual machines.

[0005] This application provides a virtual machine migration system, including: a source host machine and its corresponding first acceleration device, and a destination host machine and its corresponding second acceleration device; The first acceleration device is used to send the source address information of the last round of dirty pages in the source host to the second acceleration device during the migration process of the target virtual machine; and to read the final device status data of the target virtual machine from the source host and send it to the second acceleration device. The second acceleration device is configured to obtain the last round of dirty pages from the source host machine based on the received source address information; during the process of obtaining the last round of dirty pages, it simultaneously receives the device status data sent by the first acceleration device; and writes the last round of dirty pages and the device status data into the destination host machine.

[0006] This application embodiment also provides a virtual machine migration method, applicable to a first acceleration device corresponding to the source host machine, the method comprising: During the migration of the target virtual machine, the source address information of the last round of dirty pages is obtained from the source host. The source address information is sent to the second acceleration device corresponding to the destination host, so that the second acceleration device can obtain the last round of dirty pages from the source host based on the source address information; Read the final device status data of the target virtual machine from the source host machine; During the process of the second acceleration device acquiring the last round of dirty pages, the device status data is sent to the second acceleration device in parallel, so that the second acceleration device can write the last round of dirty pages and the device status data into the destination host machine.

[0007] This application also provides a virtual machine migration method, applicable to a second acceleration device corresponding to the destination host machine, the method comprising: During the migration of the target virtual machine, the source address information of the last round of dirty pages sent by the first acceleration device corresponding to the source host is received; The last round of dirty pages is obtained from the source host machine based on the source address information; During the process of acquiring the last round of dirty pages, the final device status data of the target virtual machine sent by the first acceleration device is received in parallel. The last round of dirty pages and the device status data are written to the destination host machine.

[0008] This application embodiment also provides an acceleration device connected to a source host machine, including a first iteration component and a first communication component; The first iterative component is used to, during the migration process of the target virtual machine, obtain the source address information of the last round of dirty pages from the source host; send the source address information to the second acceleration device corresponding to the target host; read the final device status data under the target virtual machine from the source host; and, during the process of the second acceleration device obtaining the last round of dirty pages, send the device status data to the second acceleration device in parallel, so that the second acceleration device can write the last round of dirty pages and the device status data to the target host. The first communication component is used to read the last round of dirty pages from the source host and send them to the second acceleration device, so that the second acceleration device can obtain the last round of dirty pages based on the source address information.

[0009] This application embodiment also provides an acceleration device, connected to a target host machine, including a second iteration component and a second communication component; The iterative component is used to receive the source address information of the last round of dirty pages sent by the first acceleration device corresponding to the source host during the migration process of the target virtual machine and provide it to the second communication component; during the process of obtaining the last round of dirty pages using the communication component, it simultaneously receives the final device status data of the target virtual machine sent by the first acceleration device; and writes the device status data to the target host. The second communication component is used to obtain the last round of dirty pages from the source host based on the source address information; and to write the last round of dirty pages to the destination host.

[0010] This application also provides a computer-readable storage medium for storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the aforementioned virtual machine migration method.

[0011] In this embodiment, a first acceleration device is connected to the source host and a second acceleration device is connected to the destination host. During the virtual machine migration process, the first acceleration device can send the source address information of the last round of dirty pages in the source host to the second acceleration device, which then undertakes the migration of the last round of dirty pages. After sending the source address information, the first acceleration device becomes idle and can then undertake the migration of device state data. In this way, the migration of the last round of dirty pages undertaken by the second acceleration device and the migration of device state data undertaken by the first acceleration device can be performed in parallel, without waiting for the dirty page migration to complete before performing the device state data migration. This effectively hides part or all of the time spent on device state data migration, greatly shortening the virtual machine's pause time and thus effectively improving virtual machine migration efficiency. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic diagram of the structure of a virtual machine hot migration system provided as an exemplary embodiment of this application; Figure 2 A schematic diagram of another virtual machine migration system provided as an exemplary embodiment of this application; Figure 3 A schematic diagram of the structure of another virtual machine migration system provided as an exemplary embodiment of this application; Figure 4 A flowchart illustrating a virtual machine migration method provided as another exemplary embodiment of this application; Figure 5A flowchart illustrating another virtual machine migration method provided as another exemplary embodiment of this application; Figure 6 A schematic diagram of the structure of an acceleration device is provided as another exemplary embodiment of this application; Figure 7 This is a schematic diagram of the structure of another acceleration device provided as yet another exemplary embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] Currently, during virtual machine migration, device state migration can only be performed serially after the final round of dirty page migration is completed, resulting in excessively long virtual machine pause times. To address this, some embodiments of this application involve connecting a first acceleration device to the source host and a second acceleration device to the destination host. During virtual machine migration, the first acceleration device can send the source address information of the final round of dirty pages from the source host to the second acceleration device, which then handles the migration of the final round of dirty pages. After sending the source address information, the first acceleration device becomes idle and can then handle the migration of device state data. In this way, the migration of the final round of dirty pages by the second acceleration device and the migration of device state data by the first acceleration device can be performed in parallel, eliminating the need to wait for the dirty page migration to complete before performing the device state data migration. This effectively hides part or all of the time spent on device state data migration, significantly shortening the virtual machine pause time and thus effectively improving virtual machine migration efficiency.

[0015] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0016] Figure 1 This is a schematic diagram of the structure of a virtual machine hot migration system provided as an exemplary embodiment of this application. Figure 1 As shown, the system includes: a source host machine and its corresponding first acceleration device, and a destination host machine and its corresponding second acceleration device. The source host machine refers to the host machine where the virtual machine to be migrated resides, and the destination host machine is the host machine to which the virtual machine to be migrated will be located.

[0017] During virtual machine migration, multiple iterations are required to send memory pages to the destination host. The first iteration sends all memory pages. Subsequent iterations send dirty pages (i.e., changed memory pages) from the previous pre-copy process. The final iteration is a halt copy phase, during which the migrating virtual machine is paused and its corresponding memory updates are stopped. Afterward, the dirty pages from the final round are copied to the destination host. The virtual machine migration process also involves the migration of device state data. Device state data refers to key information used to record the operating status of devices on the virtual machine. Device state data is typically stored in the device's internal registers, such as the device's PCI configuration space and BAR space. The migration of device state data also needs to be performed after the virtual machine is paused. In this embodiment, parallel migration of the final round of dirty pages and device state data can be achieved using acceleration devices connected to the source and destination hosts respectively, thus saving virtual machine pause time.

[0018] Therefore, refer to Figure 1 In this embodiment, the first acceleration device can send the source address information of the last round of dirty pages in the source host to the second acceleration device during the migration of the target virtual machine. In practical applications, the first acceleration device can read the dirty page information from the source host. The dirty page information can be used to store a flag indicating whether several memory pages allocated to the virtual machine are dirty pages. The first acceleration device can be externally connected to the source host, or it can be integrated into the source host; this embodiment does not limit the actual connection method between the two. Optionally, a dirty page bitmap can be used as the dirty page information, and the dirty page bitmap can be a binary sequence. It should be understood that the dirty page information in this embodiment can also take other information forms, and is not limited to a dirty page bitmap. For example, if a virtual machine is allocated 512KB of memory, and each memory page is 4KB in size, then the number of memory pages is 512 / 4 = 128. Therefore, the virtual machine corresponds to a 128-bit dirty page bitmap. Each bit represents whether a memory page is dirty. If the first bit of the dirty page bitmap is 1, it means the first memory page is dirty; if the second bit is 0, it means the second memory page is not dirty. It is worth noting that in this embodiment, the dirty page bitmap is existing on the source host machine and is typically maintained by KVM (Kernel-based Virtual Machine) on the source host machine.

[0019] In this way, the source host machine can offload the entire dirty page iteration process (i.e., the control plane operation) of the virtual machine migration scheme to the first acceleration device. Therefore, the first acceleration device can locate the source address information of the last round of dirty pages based on the dirty page information. In this embodiment, the first acceleration device can traverse the dirty page information, that is, scan the dirty page information bit by bit to find the source address information of the dirty pages. The source address information is used to characterize the location of the dirty page in the source host machine's memory space. Here, the dirty page iteration process is completely decoupled from the source host machine; this process has zero dependency on the resources of the source host machine, meaning that this process does not require any CPU resources from the source host machine.

[0020] The second acceleration device can obtain the last round of dirty pages from the source host based on the received source address information.

[0021] In this embodiment, a customized hot migration program can run on the destination host machine. Since there are a large number of idle resources on the destination host machine before the hot migration is completed, this embodiment can use these idle resources to support the operation of the hot migration program. For example, the destination host machine usually reserves CPU resources for the virtual machine to be migrated. In this embodiment, the hot migration program can use the CPU resources allocated for the migrated virtual machine on the destination host machine to support its operation. It should be understood that the CPU resources allocated for the migrated virtual machine are idle and available before the virtual machine migration is completed. Of course, this is only an example. In this embodiment, other idle resources on the destination host machine can also be used to support the operation of the hot migration program on it. This embodiment does not limit this. Based on this, the destination host machine can drive the connected second acceleration device based on the customized hot migration program running on it, thereby enabling the second acceleration device to undertake the dirty page migration work.

[0022] In one optional implementation: the second acceleration device can provide the received source address information to the destination host and trigger the destination host to generate a dirty page read request. The destination host can then send the dirty page read request to its corresponding second acceleration device. The second acceleration device can then read the dirty pages to be transferred in this round from the source host using the first acceleration device, according to the dirty page read request initiated by the destination host. In this implementation, the destination host can initiate a dirty page read request based on the obtained source address information, driven by a customized hot migration program. Here, the destination host can also include the destination address information allocated to the dirty pages to be transferred in this round in the dirty page read request, so that the dirty pages can be written to the correct location in the destination host's memory space after being read back. During hot migration, the addresses allocated to memory pages on the source host and the addresses reserved for memory pages on the destination host are usually in one-to-one correspondence. Therefore, upon receiving the source address information, the destination host can easily and quickly find the destination address corresponding to the dirty page based on this correspondence and include it in the dirty page read request.

[0023] In this implementation, the second acceleration device can retrieve the last round of dirty pages from the source host using Remote Direct Memory Access (RDMA). Correspondingly, the destination host can generate an RDMA instruction as a dirty page read request based on the source address information and the destination address information allocated to the last round of dirty pages on the destination host, enabling the second acceleration device to retrieve the last round of dirty pages from the source host using RDMA. Of course, besides using RDMA, the second acceleration device can also use other communication methods to implement dirty page migration. For example, TCP can be used. In TCP mode, the second acceleration device can directly read the last round of dirty pages from the source host using DMA or similar methods, and receive the last round of dirty pages read by the first acceleration device via TCP. Then, the second acceleration device can directly write the last round of dirty pages to the destination host via DMA or similar methods. This embodiment does not limit the communication method used in the dirty page migration process; appropriate hardware can be used to implement the second acceleration device for different communication methods.

[0024] In this way, the dirty page migration work has been handed over to the second acceleration device, and the first acceleration device is now idle.

[0025] The idle first acceleration device can read the final device status data of the target virtual machine from the source host and send it to the second acceleration device. This final device status data records the final running state of the devices on the target virtual machine after it has been paused. During this process, the first acceleration device can read the final device status data of the target virtual machine from the source host using direct memory access (DMA) / network methods. This embodiment does not limit the method by which the first acceleration device obtains the final device status data from the source host. Furthermore, in practical applications, the first acceleration device can, after reading all device states of the target virtual machine from the source host, package the read device states (corresponding to the device status data in this document) and send them to the second acceleration device. Thus, the device status data can contain one or more device states, and the device status data can use device identifiers as indexes; that is, in the device status data, a single device identifier can be associated with one or more device states.

[0026] In this way, the process of retrieving the last round of dirty pages from the source host and the process of retrieving the device status from the source host can be parallel. Furthermore, the second acceleration device can receive device status data sent by the first acceleration device in parallel during the process of retrieving the last round of dirty pages. That is, the process of the first acceleration device sending device status data to the second acceleration device and the process of the second acceleration device retrieving the last round of dirty pages from the source host can also be parallel.

[0027] Therefore, in this embodiment, the migration of the last round of dirty pages undertaken by the second acceleration device and the migration of device status data undertaken by the first acceleration device can be performed in parallel.

[0028] Continue to refer to Figure 1 The second acceleration device can obtain the last round of dirty pages and the final device state data corresponding to the target virtual machine in parallel. The second acceleration device can then write the obtained last round of dirty pages and device state data to the target host machine.

[0029] The second acceleration device can write the last round of dirty pages and the final device state data to the destination host machine using Direct Memory Access (DMA). As mentioned earlier, the dirty page read request initiated by the destination host machine carries the destination address information of the dirty page on the destination host machine. Therefore, the second acceleration device can write the received dirty pages to be transferred in this round to the corresponding address on the destination host machine according to the destination address information in the dirty page read request. In this way, the destination host machine can receive the last round of dirty pages written by its corresponding second acceleration device after issuing a dirty page read request, thereby completing the dirty page migration. Similarly, the second acceleration device can also write the received device state data to the destination host machine, thereby completing the migration of device state data.

[0030] In summary, in this embodiment, a first acceleration device can be connected to the source host and a second acceleration device can be connected to the destination host. During the virtual machine migration process, the first acceleration device can send the source address information of the last round of dirty pages in the source host to the second acceleration device, which then undertakes the migration of the last round of dirty pages. After sending the source address information, the first acceleration device becomes idle and can then undertake the migration of device state data. In this way, the migration of the last round of dirty pages undertaken by the second acceleration device and the migration of device state data undertaken by the first acceleration device can be executed in parallel, without waiting for the dirty page migration to complete before performing the device state data migration. This effectively hides part or all of the time spent on device state data migration, greatly shortening the virtual machine's pause time and thus effectively improving the virtual machine migration efficiency.

[0031] In the above or following embodiments, during the process of the second acceleration device writing the last round of dirty page and device state data to the destination host, both dirty page migration and device state migration may modify the virtual machine memory. Furthermore, even after the virtual machine is paused, some device states may still change. Therefore, the parallel migration method may encounter out-of-order issues, leading to incorrect memory data after migration. This is why traditional solutions require device state migration to be performed only after the last round of dirty page migration is completed.

[0032] To avoid out-of-order issues that may result from parallel migration, in this embodiment, the second acceleration device can identify the device state corresponding to the specified device from the device state data. The specified device needs to store its device state in memory. After writing all the dirty pages in the last round to the destination host machine, the identified device state is then written to the destination host machine.

[0033] As mentioned earlier, device identifiers can be used as indexes in device status data. Based on this, in one exemplary scheme, the second acceleration device can search for device statuses associated with any specified device identifier from the device status data based on pre-configured specified device identifiers, thereby identifying the device status corresponding to the specified device. The specified devices can be determined based on experience; they typically require memory for storing device statuses, and the number of specified devices is finite and exhaustive. Furthermore, the pre-configured specified device identifiers in the second acceleration device can be increased or decreased according to technological advancements in the field.

[0034] Thus, in this embodiment, after receiving the device status data sent by the first acceleration device, the second acceleration device can skip some of the device status data and proceed with writing the remaining device status data to the target host machine. Here, skipping can be understood as pausing the writing process. The second acceleration device can write the skipped device status data to the target host machine only after writing all the dirty pages from the last round. This prevents the device status corresponding to a specified device from being incorrectly overwritten by the dirty pages from the last round on the target host machine side.

[0035] For example, if the device state corresponding to device A on the virtual machine modifies dirty page B in the last round of dirty pages, but the process of the second acceleration device retrieving dirty page B from the source host is delayed, causing dirty page B to be written to the destination host after the device state corresponding to device A, then dirty page B will overwrite the device state corresponding to device A, resulting in a memory error. In the order-preserving scheme provided in this embodiment, the second acceleration device will temporarily skip the device state corresponding to device A and write the device state corresponding to device A to the destination host only after all dirty pages in the last round (including dirty page B) have been written. This ensures the correctness of the device state corresponding to device A obtained on the destination host and avoids memory errors.

[0036] In some possible design schemes, a sequence-preserving controller can be set in the second acceleration device to implement the sequence-preserving logic in this embodiment, ensuring that the device state of the specified device is written to the destination host after the relevant dirty pages. Additionally, the second acceleration device can use queues to manage the received device state data, and the destination host can subscribe to these queues to write the device state. These are merely examples, and this embodiment is not limited thereto.

[0037] Accordingly, in this embodiment, an order preservation mechanism can be introduced in the second acceleration device to temporarily skip the device state corresponding to the specified device from the received device state data, and write the skipped device state to the destination host after all the dirty pages in the last round are written, so as to avoid the device state of the specified device being incorrectly overwritten by the dirty pages in the last round, thereby avoiding out-of-order problems that may occur due to concurrent migration.

[0038] Figure 2 This is a schematic diagram of another virtual machine migration system provided as an exemplary embodiment of this application. (Reference) Figure 2 In the above or below embodiments, a first communication link and a second communication link are provided in parallel between the first acceleration device and the second acceleration device. The first communication link is used by the second acceleration device to read the last round of dirty pages from the source host, and the second communication link is used by the first acceleration device to send device status data to the second acceleration device.

[0039] In this embodiment, two parallel communication links can be set up between the first acceleration device and the second acceleration device. The two communication links can be used for the migration of dirty pages in the last round and the migration of device status data, respectively, thereby supporting parallel migration.

[0040] The second communication link allows the first acceleration component to send the source address information of the last round of dirty pages to the second acceleration device. After sending the source address information, the second communication link becomes idle and can then be used for device state migration. The first communication link allows the second acceleration device to read the last round of dirty pages from the source host. In this embodiment, the second acceleration device can read the last round of dirty pages from the source host via the first communication link and the first acceleration device. It should be understood that although the first acceleration device assists in the dirty page migration, the first and second communication links are independent of each other. Therefore, the dirty page migration assisted by the first acceleration device based on the first communication link will not compete for resources with the device state data migration based on the second communication link. The two are independent and do not interfere with each other.

[0041] refer to Figure 2 In one optional hardware implementation: the first acceleration device may include a first RDMA component and a first iteration component, and the second acceleration device may include a second RDMA component and a second iteration component. A first communication link is established between the first RDMA component and the second RDMA component, and a second communication link is established between the first iteration component and the second iteration component. That is, the first acceleration device and the second acceleration device each configure dedicated communication components for their respective communication links, so that the two communication links can operate independently without interfering with each other. Of course, in this embodiment, other implementation methods can also be used to ensure the independence of the two communication links, and it is not limited to this.

[0042] Based on the internal structure of the acceleration device proposed in this implementation method, and referring to Figure 2 The second RDMA component can cooperate with the first RDMA component to respond to dirty page read requests initiated by the destination host. Based on this, the first RDMA component can read the last round of dirty pages from the source host and return it to the second RDMA component; the second RDMA component can then write the last round of dirty pages to the destination host. The first RDMA component can read the last round of dirty pages from the source host using a direct memory access (DMA) / network method. This DMA method allows the first acceleration device to successfully read the last round of dirty pages from the source host without relying on the CPU resources of the source host. In this way, the dirty page transfer process is completely decoupled from the source host, and this process has zero dependency on the resources of the source host; that is, this process does not require any CPU resources from the source host.

[0043] In addition, the first RDMA component and the second RDMA component are implemented using FPGA, RDMA network card or other devices with RDMA capability based on ASIC. Here, there is no limitation on the hardware type of the RDMA component.

[0044] Continue to refer to Figure 2 The first iteration component can send the source address information of the last round of dirty pages to the second iteration component via the second communication link. After completing this part of the work, it obtains device status data from the source host. The first and second iteration components are implemented using a system-on-a-chip (SoC), a physical network interface card (NIC), or a dedicated host. An SoC, also known as a system-on-a-chip, is a product, a dedicated integrated circuit containing a complete system and embedded software. Therefore, it is particularly suitable for the complex but computationally low dirty page iteration process in this embodiment. The SoC chip can effectively complete the dirty page information traversal during the dirty page iteration process. When using a dedicated host to implement the iteration components, multiple source hosts can share the same dedicated host. This dedicated host can provide dirty page iteration support for different source hosts. For example, the dedicated host can run different dirty page iteration processes for different source hosts to isolate the dirty page iteration work corresponding to different source hosts. The working logic of each dirty page iteration process can be referred to in the previous description of the dirty page iteration process in the first acceleration device, and will not be repeated here. If multiple network cards are configured on the source host, one of them can be used as an iterative component in the acceleration device. The network card can perform related tasks by customizing the relevant processing logic of the aforementioned iterative component in the network card.

[0045] It is worth noting that when implementing the iterative components in hardware forms such as SOC chips and dedicated host machines, the aforementioned RDMA component can be used as a hardware bridge between the iterative components and the source / destination host machine. In this way, the first iterative component can use the first RDMA component as a hardware bridge (e.g., an FPGA channel) to read dirty page information from the source host machine, and the second iterative component and the destination host machine can use the second RDMA component as a hardware bridge to transfer the address information of the last round of dirty pages, etc. Of course, this is merely an example, and this embodiment is not limited to it.

[0046] Accordingly, in this embodiment, two parallel communication links can be set between the first acceleration device and the second acceleration device, so as to support the parallel execution of the last round of dirty page migration and the device status data migration, which are independent of each other and do not interfere with each other.

[0047] Figure 3 This is a schematic diagram of the structure of another virtual machine migration system provided as an exemplary embodiment of this application. (Reference) Figure 3In the above or below embodiments, a first type of transmission queue and a second type of transmission queue may be set in the second acceleration device; the second acceleration device uses the first type of transmission queue to read the last type of dirty page from the source host, and the second acceleration device uses the second type of transmission queue to receive device status data sent by the first acceleration device.

[0048] Compared to Figure 2 In this embodiment of the virtual machine migration system, an alternative implementation scheme is provided to support the independence between the final round of dirty page migration and the device state data migration. That is, a queue mechanism is used to isolate the final round of dirty page migration and the device state data migration.

[0049] In this embodiment, virtualization technology can be used to simulate two virtual devices in the second acceleration device and configure transmission queues for these virtual devices, thereby generating two types of transmission queues in the second acceleration device. These two types of transmission queues are isolated from each other and do not interfere with each other, thus supporting the independence between the final round of dirty page migration and the migration of device state data.

[0050] Accept Figure 2 In the exemplary hardware implementation of the acceleration device provided, in this embodiment, two types of transfer queues can be implemented in the second RDMA component of the second acceleration device. Additionally, to cooperate with the second acceleration device, the first acceleration device can also generate two types of transfer queues according to the same virtualization method. In this way, the first and second acceleration devices can support the independence between the last round of dirty page migration and the migration of device state data through the cooperation of similar transfer queues.

[0051] In addition, in this embodiment, the receiving Figure 2 In the hardware implementation of the first acceleration device, the first iteration component can read the last round of dirty pages and / or device status data from the source host and then pass it to the first RDMA component. Alternatively, the first RDMA component can read the last round of dirty pages and / or device status data from the source host. This embodiment does not limit this, as long as it ensures that the RDMA components contained in the first and second acceleration devices perform the transmission of the last round of dirty pages and device status data according to the queue mechanism.

[0052] Accordingly, in this embodiment, by introducing a queue mechanism, the migration of the last round of dirty pages and the migration of device status data can be performed in parallel, independently and without interference.

[0053] Figure 4 This is a flowchart illustrating a virtual machine migration method provided as another exemplary embodiment of this application. The method can be implemented by a first acceleration device in the aforementioned system embodiments, and the first acceleration device is connected to the source host machine. (Reference) Figure 4 The method may include: Step 400: During the migration of the target virtual machine, obtain the source address information of the last round of dirty pages from the source host. Step 401: Send the source address information to the second acceleration device corresponding to the destination host, so that the second acceleration device can obtain the last round of dirty pages from the source host based on the source address information; Step 402: Read the final device status data of the target virtual machine from the source host machine; Step 403: During the process of the second acceleration device acquiring the last round of dirty pages, the device status data is sent to the second acceleration device in parallel so that the second acceleration device can write the last round of dirty pages and device status data to the destination host machine.

[0054] In one optional embodiment, the second acceleration device identifies the device state corresponding to the specified device from the device state data. The specified device needs to store the device state using memory. After writing all the dirty pages of the last round to the destination host machine, the identified device state is then written to the destination host machine.

[0055] In an optional embodiment, the second acceleration device may search for the device status associated with any specified device identifier from the device status data based on a pre-configured specified device identifier.

[0056] In one optional embodiment, a first communication link and a second communication link are provided in parallel between the first acceleration device and the second acceleration device. The first communication link is used by the second acceleration device to read the last round of dirty pages from the source host, and the second communication link is used by the first acceleration device to send device status data to the second acceleration device.

[0057] In one optional embodiment, the first acceleration device includes a first RDMA component and a first iteration component, and the second acceleration device includes a second RDMA component and a second iteration component. A first communication link is established between the first RDMA component and the second RDMA component, and a second communication link is established between the first iteration component and the second iteration component.

[0058] In one optional embodiment, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities; the first iteration component and the second iteration component are implemented using a system-on-a-chip (SoC), a physical network card, or a dedicated host machine.

[0059] In one optional embodiment, the second acceleration device is provided with a first type of transmission queue and a second type of transmission queue; the second acceleration device uses the first type of transmission queue to read the last type of dirty pages from the source host, and the second acceleration device uses the second type of transmission queue to receive device status data sent by the first acceleration device.

[0060] In an alternative embodiment, step 400 may include: The final device status data of the target virtual machine is read from the source host machine using the Direct Memory Access (DMA) / Network method. The second acceleration device writes the last round of dirty pages and device status data to the destination host machine using direct memory access (DMA) / network methods.

[0061] It is worth noting that the technical details of the above embodiments of the virtual machine migration method can be found in the description of the first acceleration device in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0062] Figure 5 This is a flowchart illustrating another virtual machine migration method provided as an exemplary embodiment of this application. The method can be implemented by a second acceleration device in the aforementioned system embodiments, and the second acceleration device is connected to the target host machine. (See reference...) Figure 5 The method may include: Step 500: During the migration of the target virtual machine, receive the source address information of the last round of dirty pages sent by the first acceleration device corresponding to the source host. Step 501: Obtain the last round of dirty pages from the source host machine based on the source address information; Step 502: During the process of acquiring the last round of dirty pages, the final device status data of the target virtual machine sent by the first acceleration device is received in parallel. Step 503: Write the last round of dirty pages and device status data to the destination host machine.

[0063] In an alternative embodiment, step 503 may include: Identify the device status of a specified device from the device status data. The specified device needs to store its device status in memory. After writing all the dirty pages in the last round to the destination host machine, the identified device status is then written to the destination host machine.

[0064] In an optional embodiment, the step of identifying the device status corresponding to a specified device from the device status data may include: Based on a pre-configured specified device identifier, the device status associated with any specified device identifier is found from the device status data.

[0065] In one optional embodiment, a first communication link and a second communication link are provided in parallel between the first acceleration device and the second acceleration device. The first communication link is used by the second acceleration device to read the last round of dirty pages from the source host, and the second communication link is used by the first acceleration device to send device status data to the second acceleration device.

[0066] In one optional embodiment, the first acceleration device includes a first RDMA component and a first iteration component, and the second acceleration device includes a second RDMA component and a second iteration component. A first communication link is established between the first RDMA component and the second RDMA component, and a second communication link is established between the first iteration component and the second iteration component.

[0067] In one optional embodiment, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities; the first iteration component and the second iteration component are implemented using a system-on-a-chip (SoC), a physical network card, or a dedicated host machine.

[0068] In one optional embodiment, the second acceleration device is provided with a first type of transmission queue and a second type of transmission queue; the second acceleration device uses the first type of transmission queue to read the last type of dirty pages from the source host, and the second acceleration device uses the second type of transmission queue to receive device status data sent by the first acceleration device.

[0069] In an optional embodiment, the first acceleration device may read the final device status data of the target virtual machine from the source host machine in the manner of direct memory access DMA / network; Step 503 may include: writing the last round of dirty pages and device state data to the destination host in a direct memory access DMA / network manner.

[0070] It is worth noting that the technical details of the above embodiments of the virtual machine migration method can be found in the description of the second acceleration device in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0071] It should be noted that the execution subject of each step in the method provided in the above embodiments can be the same component, or the method can be executed by different components. For example, the execution subject of steps 500 to 503 can be component A; or the execution subject of steps 500 and 802 can be component A, and the execution subject of step 502 can be component B; and so on.

[0072] Furthermore, in some processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 500, 502, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different components, links, devices, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0073] Figure 6 This is a schematic diagram of an acceleration device provided as another exemplary embodiment of this application. The acceleration device can be connected to a source host machine. Figure 6 As shown, the acceleration device includes: a first iteration component 61 and a first communication component 62; The first iteration component 61 is used to, during the migration process of the target virtual machine, obtain the source address information of the last round of dirty pages from the source host; send the source address information to the second acceleration device corresponding to the target host; read the final device status data under the target virtual machine from the source host; and, during the process of the second acceleration device obtaining the last round of dirty pages, send the device status data to the second acceleration device in parallel, so that the second acceleration device can write the last round of dirty pages and the device status data to the target host. The first communication component 62 is used to read the last round of dirty pages from the source host and send them to the second acceleration device, so that the second acceleration device can obtain the last round of dirty pages based on the source address information.

[0074] In one optional embodiment, the second acceleration device identifies the device state corresponding to the specified device from the device state data. The specified device needs to store the device state using memory. After writing all the dirty pages of the last round to the destination host machine, the identified device state is then written to the destination host machine.

[0075] In an optional embodiment, the second acceleration device may search for the device status associated with any specified device identifier from the device status data based on a pre-configured specified device identifier.

[0076] In one optional embodiment, a first communication link and a second communication link are provided in parallel between the first acceleration device and the second acceleration device. The first communication link is used by the second acceleration device to read the last round of dirty pages from the source host, and the second communication link is used by the first acceleration device to send device status data to the second acceleration device.

[0077] In an optional embodiment, the first communication component 62 may include a first RDMA component, the second acceleration device includes a second RDMA component and a second iteration component, a first communication link is established between the first RDMA component and the second RDMA component, and a second communication link is established between the first iteration component and the second iteration component.

[0078] In one optional embodiment, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities; the first iteration component and the second iteration component are implemented using a system-on-a-chip (SoC), a physical network card, or a dedicated host machine.

[0079] In one optional embodiment, the second acceleration device is provided with a first type of transmission queue and a second type of transmission queue; the second acceleration device uses the first type of transmission queue to read the last type of dirty pages from the source host, and the second acceleration device uses the second type of transmission queue to receive device status data sent by the first acceleration device.

[0080] In an optional embodiment, the iterative component 61, during the process of obtaining the source address information of the last round of dirty pages from the source host, can be used to: The final device status data of the target virtual machine is read from the source host machine using the Direct Memory Access (DMA) / Network method. The second acceleration device writes the last round of dirty pages and device status data to the destination host machine using direct memory access (DMA) / network methods.

[0081] It is worth noting that the technical details of the various embodiments of the acceleration device described above can be found in the relevant description of the first acceleration device in the foregoing system embodiments. To save space, these details will not be repeated here, but this should not diminish the scope of protection of this application. Furthermore, Figure 6 The diagram only shows some components and does not mean that the acceleration device includes only these components. Figure 6 The components shown.

[0082] Figure 7 This is a schematic diagram of another acceleration device provided as a further exemplary embodiment of this application. This acceleration device can be connected to a target host machine. Figure 7 As shown, the acceleration device includes: a second iteration component 71 and a second communication component 72; The second iteration component 71 is used to receive the source address information of the last round of dirty pages sent by the first acceleration device corresponding to the source host during the migration process of the target virtual machine and provide it to the second communication component; during the process of obtaining the last round of dirty pages using the communication component, it receives the final device status data under the target virtual machine sent by the first acceleration device in parallel; and writes the device status data to the target host. The second communication component 72 is used to obtain the last round of dirty pages from the source host based on the source address information; and to write the last round of dirty pages to the destination host.

[0083] In an optional embodiment, the second iteration component 71, during the process of writing device status data to the destination host, can be used to: Identify the device status of a specified device from the device status data. The specified device needs to store its device status in memory. After writing all the dirty pages in the last round to the destination host machine, the identified device status is then written to the destination host machine.

[0084] In an optional embodiment, the second iteration component 71, during the process of identifying the device state corresponding to a specified device from the device state data, can be used to: Based on a pre-configured specified device identifier, the device status associated with any specified device identifier is found from the device status data.

[0085] In one optional embodiment, a first communication link and a second communication link are provided in parallel between the first acceleration device and the second acceleration device. The first communication link is used by the second acceleration device to read the last round of dirty pages from the source host, and the second communication link is used by the first acceleration device to send device status data to the second acceleration device.

[0086] In one optional embodiment, the first acceleration device includes a first RDMA component and a first iteration component, the second communication component 72 in the second acceleration device includes a second RDMA component, a first communication link is established between the first RDMA component and the second RDMA component, and a second communication link is established between the first iteration component and the second iteration component 71.

[0087] In one optional embodiment, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities; the first iteration component and the second iteration component 71 are implemented using a system-on-a-chip (SoC), a physical network card, or a dedicated host machine.

[0088] In one optional embodiment, the second acceleration device is provided with a first type of transmission queue and a second type of transmission queue; the second acceleration device uses the first type of transmission queue to read the last type of dirty pages from the source host, and the second acceleration device uses the second type of transmission queue to receive device status data sent by the first acceleration device.

[0089] In an optional embodiment, the first acceleration device may read the final device status data of the target virtual machine from the source host machine in the manner of direct memory access DMA / network; The second iteration component 71 and the second communication component 72 can write the last round of dirty pages and device status data to the destination host in a direct memory access DMA / network manner.

[0090] It is worth noting that the technical details of the various embodiments of the acceleration device mentioned above can be found in the description of the second acceleration device in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0091] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by the first acceleration device / second acceleration device in the above method embodiments.

[0092] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0096] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0097] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0098] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0099] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0100] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A virtual machine migration system, comprising: The source host machine and its corresponding first acceleration device, and the destination host machine and its corresponding second acceleration device; The first acceleration device is used to send the source address information of the last round of dirty pages in the source host to the second acceleration device during the migration process for the target virtual machine; Read the final device status data of the target virtual machine from the source host machine and send it to the second acceleration device; The second acceleration device is used to obtain the last round of dirty pages from the source host based on the received source address information; During the process of acquiring the last round of dirty pages, the device status data sent by the first acceleration device is received in parallel; the last round of dirty pages and the device status data are written to the destination host machine.

2. The system according to claim 1, wherein the second acceleration device, during the process of writing the last round of dirty pages and the device status data to the destination host, is used to: From the device status data, identify the device status corresponding to the specified device. The specified device needs to store the device status in memory. After writing all the dirty pages of the last round to the target host machine, the identified device status is then written to the target host machine.

3. The system according to claim 2, wherein the second acceleration device, in the process of identifying the device state corresponding to the specified device from the device state data, is used to: Based on a pre-configured specified device identifier, the device status associated with any specified device identifier is searched from the device status data.

4. The system according to any one of claims 1-3, wherein a first communication link and a second communication link are provided in parallel between the first acceleration device and the second acceleration device, the first communication link being used by the second acceleration device to read the last round of dirty pages from the source host, and the second communication link being used by the first acceleration device to send the device status data to the second acceleration device.

5. The system according to claim 4, wherein the first acceleration device includes a first RDMA component and a first iteration component, the second acceleration device includes a second RDMA component and a second iteration component, a first communication link is established between the first RDMA component and the second RDMA component, and a second communication link is established between the first iteration component and the second iteration component.

6. In the system according to claim 5, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities; the first iteration component and the second iteration component are implemented using a system-level SOC chip, a physical network card, or a dedicated host machine.

7. The system according to any one of claims 1-3, wherein the second acceleration device is provided with a first type of transmission queue and a second type of transmission queue; the second acceleration device uses the first type of transmission queue to read the last round of dirty pages from the source host, and the second acceleration device uses the second type of transmission queue to receive the device status data sent by the first acceleration device.

8. The system according to any one of claims 1-3, wherein the first acceleration device, during the process of reading the final device status data under the target virtual machine from the source host, is used to: The final device status data of the target virtual machine is read from the source host machine using either Direct Memory Access (DMA) or network access. During the process of writing the last round of dirty pages and the device status data to the destination host, the second acceleration device is used to: The last round of dirty pages and the device status data are written to the destination host machine using either Direct Memory Access (DMA) or network methods.

9. A virtual machine migration method, applicable to a first acceleration device corresponding to a source host machine, the method comprising: During the migration of the target virtual machine, the source address information of the last round of dirty pages is obtained from the source host. The source address information is sent to the second acceleration device corresponding to the destination host, so that the second acceleration device can obtain the last round of dirty pages from the source host based on the source address information; Read the final device status data of the target virtual machine from the source host machine; During the process of the second acceleration device acquiring the last round of dirty pages, the device status data is sent to the second acceleration device in parallel, so that the second acceleration device can write the last round of dirty pages and the device status data into the destination host machine.

10. A virtual machine migration method, applicable to a second acceleration device corresponding to a destination host machine, the method comprising: During the migration of the target virtual machine, the source address information of the last round of dirty pages sent by the first acceleration device corresponding to the source host is received; The last round of dirty pages is obtained from the source host machine based on the source address information; During the process of acquiring the last round of dirty pages, the final device status data of the target virtual machine sent by the first acceleration device is received in parallel. The last round of dirty pages and the device status data are written to the destination host machine.

11. An acceleration device, connected to a source host machine, comprising a first iteration component and a first communication component; The first iterative component is used to obtain the source address information of the last round of dirty pages from the source host during the migration process for the target virtual machine; The source address information is sent to the second acceleration device corresponding to the destination host machine; Read the final device status data of the target virtual machine from the source host machine; During the process of the second acceleration device acquiring the last round of dirty pages, the device status data is sent to the second acceleration device in parallel, so that the second acceleration device can write the last round of dirty pages and the device status data into the destination host machine; The first communication component is used to read the last round of dirty pages from the source host and send them to the second acceleration device, so that the second acceleration device can obtain the last round of dirty pages based on the source address information.

12. An acceleration device, connected to a target host machine, comprising a second iteration component and a second communication component; The second iteration component is used to receive the source address information of the last round of dirty pages sent by the first acceleration device corresponding to the source host during the migration process of the target virtual machine and provide it to the second communication component; during the process of obtaining the last round of dirty pages using the second communication component, it simultaneously receives the final device status data of the target virtual machine sent by the first acceleration device. Write the device status data to the target host machine; The second communication component is used to obtain the last round of dirty pages from the source host based on the source address information; The last round of dirty pages is written to the target host machine.

13. A computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the virtual machine migration method of any one of claims 9 or 10.

Images