A method, device, system and storage medium for live migration of a virtual machine

Abstract

Embodiments of the present application provide a virtual machine live migration method, device, system and storage medium. An optimization scheme is proposed for a traditional live migration architecture. By configuring an acceleration device for a source host, the live migration work of the control plane and the data plane can be completely sunk to the acceleration device for execution on the source host side. The live migration work can be completed by cooperation of the destination host and the acceleration device, which realizes zero dependence of the live migration work on the source host resources. Even in the case that the computing resources of the source host are fully occupied, the performance of the live migration is not affected, and the live migration function achieves zero jitter for a virtual machine running on the source host.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to a method, device, system and storage medium for hot migration of virtual machines. Background Technology

[0002] In public clouds, there are many large-scale virtual machines. Due to their high configuration and heavy load, these virtual machines have always been a challenge for hot migration in the industry, mainly in the following two aspects:

[0003] 1. Large memory usage and extremely high dirty page rate make it difficult for hot migration to achieve iterative convergence under typical network conditions.

[0004] 2. There are many instances on the physical CPU resources, especially when the instances have already filled the entire machine, leaving no spare physical CPU resources for hot migration.

[0005] However, all current hot migration methods in the industry require CPU resources to be consumed in the virtual machine's runtime environment for hot migration, which means that hot migration and VCPU have to share physical CPU resources. This will cause the virtual machine to jitter, and on the other hand, the migration performance will be poor because hot migration does not get enough computing resources. Summary of the Invention

[0006] This application provides a method, device, system, and storage medium for hot migration of virtual machines, in order to improve the resource consumption problem of the source host machine during hot migration of virtual machines.

[0007] This application provides a virtual machine hot migration system, including: a source host machine and its corresponding first acceleration device, and a destination host machine;

[0008] The first acceleration device is used to read dirty page information from the source host during hot migration; find the source address information of the dirty page to be transferred in this round based on the dirty page information; and send the source address information to the destination host.

[0009] The destination host machine is used to obtain the source address information; initiate a dirty page read request based on the source address information; and use the first acceleration device to obtain the dirty pages to be transferred in this round from the source host machine.

[0010] This application embodiment also provides a method for hot migration of virtual machines, applicable to a first acceleration device configured for a source host machine, the method comprising:

[0011] During the hot migration process, dirty page information is read from the source host machine;

[0012] Based on the dirty page information, locate the source address information of the dirty pages to be transferred in this round;

[0013] The source address information is sent to the destination host machine so that the destination host machine can initiate a dirty page read request based on the source address information;

[0014] Read the dirty pages to be transferred in this round from the source host machine;

[0015] The dirty pages to be transferred in this round are provided to the destination host in response to the dirty page read request.

[0016] This application also provides a method for hot migration of a virtual machine, applicable to a second acceleration device configured for a destination host machine, the method comprising:

[0017] During hot migration, the source address information of the dirty pages to be transferred in this round is received from the first acceleration device configured for the source host.

[0018] Provide the source address information to the destination host machine;

[0019] According to the dirty page read request initiated by the destination host based on the source address information, the first acceleration device is used to read the dirty pages to be transferred in this round from the source host;

[0020] The acquired dirty pages to be transferred in this round are written into the target host machine.

[0021] This application embodiment also provides an acceleration device connected to a source host machine, including a dirty page iteration component and a communication component;

[0022] The dirty page iteration component is used to read dirty page information from the source host during hot migration; find the source address information of the dirty page to be transferred in this round based on the dirty page information; and send the source address information to the destination host so that the destination host can initiate a dirty page read request based on the source address information.

[0023] The communication component is used to read the dirty pages to be transferred in the current round from the source host machine; and to send the read dirty pages to be transferred in the current round to the destination host machine in response to the dirty page read request.

[0024] This application embodiment also provides an acceleration device, connected to a target source host machine, including a dirty page iteration component and a communication component;

[0025] The dirty page iteration component is used to receive, during the hot migration process, the source address information of the dirty pages to be transferred in this round provided by the first acceleration device configured for the source host; and to provide the source address information to the destination host.

[0026] The communication component is used to read the dirty pages to be transferred in this round from the source host according to the dirty page read request initiated by the destination host based on the source address information, using the first acceleration device; and to write the obtained dirty pages to be transferred in this round into the destination host.

[0027] This application also provides a computer-readable storage medium for storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the aforementioned virtual machine hot migration method.

[0028] In this embodiment, acceleration devices are respectively installed on the source host and the destination host. On the source host side, the acceleration device reads dirty page information from the source host during the hot migration process; based on the dirty page information, it finds the source address information of the dirty pages to be transferred in this round; and sends the source address information to the destination host side. On the destination host side, the source address information can be obtained; a dirty page read request is initiated based on the source address information, and the acceleration device on the source host side reads the dirty pages to be transferred in this round from the source host. Accordingly, this embodiment provides an optimization scheme for the traditional hot migration architecture. By configuring acceleration devices on the source host side, the hot migration work of the control plane and data plane can be completely offloaded to the acceleration device on the source host side. The hot migration work can be completed by the cooperation of the destination host and the acceleration device. This achieves zero dependence of the hot migration work on the resources of the source host. Even if the computing resources of the source host are fully occupied, the performance of the hot migration is not affected, and the hot migration function achieves zero jitter for the virtual machines running on the source host. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1a A schematic diagram of the structure of a virtual machine hot migration system provided as an exemplary embodiment of this application;

[0031] Figure 1b A schematic diagram of the structure of another virtual machine hot migration system provided as an exemplary embodiment of this application;

[0032] Figure 2 A schematic diagram illustrating a hardware implementation of an acceleration device provided for an exemplary embodiment of this application;

[0033] Figure 3 A flowchart illustrating a method for hot migration of a virtual machine, provided as another exemplary embodiment of this application;

[0034] Figure 4 A flowchart illustrating another virtual machine hot migration method provided as another exemplary embodiment of this application;

[0035] Figure 5 A schematic diagram of the structure of an acceleration device is provided as another exemplary embodiment of this application;

[0036] Figure 6 This is a schematic diagram of the structure of another acceleration device provided as yet another exemplary embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Currently, hot migration of virtual machines requires resources on the source host, resulting in poor migration efficiency and potential virtual machine jitter. To address this, some embodiments of this application include: acceleration devices on both the source and destination hosts; on the source host side, the acceleration device reads dirty page information from the source host during hot migration; based on the dirty page information, it locates the source address information of the dirty pages to be transferred in this round; and sends the source address information to the destination host side; while on the destination host side, the source address information can be obtained; a dirty page read request is initiated based on the source address information, and the acceleration device on the source host side reads the dirty pages to be transferred in this round from the source host. Accordingly, this application provides an optimization scheme for the traditional hot migration architecture. By configuring an acceleration device on the source host side, the hot migration work of the control plane and data plane can be carried out on the acceleration device. The hot migration work can be completed by the cooperation of the destination host and the acceleration device. This achieves zero dependence of the hot migration work on the resources of the source host. Even if the computing resources of the source host are fully occupied, the performance of hot migration will not be affected. Moreover, the hot migration function achieves zero jitter for the virtual machines running on the source host.

[0039] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0040] Figure 1a This is a schematic diagram of the structure of a virtual machine hot migration system provided as an exemplary embodiment of this application. Figure 1a As shown, the system includes: a source host machine and its corresponding first acceleration device, and a destination host machine.

[0041] The virtual machine hot migration system provided in this embodiment inherits and improves upon the traditional pre-copy hot migration scheme. In the pre-copy hot migration scheme, the source host iterates through the process of sending memory data to the destination host. The first iteration sends all memory data; subsequent iterations send dirty pages from the previous pre-copy process; and the final iteration is a halt copy phase where the source host is suspended, memory updates are stopped, and all dirty pages are copied to the destination host. The virtual machine migration system provided in this embodiment improves both the dirty page iteration process and the dirty page transfer process in the pre-copy hot migration scheme.

[0042] In this embodiment, the source host machine refers to the host machine where the virtual machine to be migrated is located, and the destination host machine is the host machine to which the virtual machine to be migrated will be migrated.

[0043] As mentioned earlier, the pre-copy hot migration scheme requires multiple rounds of migration. For ease of description, this embodiment will explain the technical solution from the perspective of one round of migration. It should be understood that the same technical solution can be used to achieve migration in each round of migration. The first round of migration in the pre-copy hot migration scheme is particularly special. In the first round of migration, all memory pages in the dirty page information can be regarded as dirty pages without performing a dirty page lookup operation. Based on this, all memory data in the source host machine can be migrated to the destination host machine according to the technical solution provided in this embodiment.

[0044] refer to Figure 1aIn this embodiment, the first acceleration device configured for the source host can read dirty page information from the source host during hot migration. The dirty page information can be used to store markers indicating whether several memory pages allocated to the virtual machine are dirty pages. The first acceleration device can be externally connected to the source host, or it can be integrated within the source host; this embodiment does not limit the actual connection method between the two. Optionally, a dirty page bitmap can be used as the dirty page information, and the dirty page bitmap can be a binary sequence. It should be understood that the dirty page information in this embodiment can also take other information forms, and is not limited to a dirty page bitmap. For example, if 512KB of memory is allocated to the virtual machine, and each memory page is 4KB in size, then the number of memory pages is 512 / 4 = 128. Therefore, the virtual machine corresponds to a 128-bit dirty page bitmap, where each bit represents whether a memory page is dirty. If the first bit of the dirty page bitmap is 1, it means the first memory page is dirty; if the second bit of the dirty page bitmap is 0, it means the second memory page is not dirty. It is worth noting that in this embodiment, the dirty page bitmap is an existing one on the source host machine, and can usually be maintained by KVM (Kernel-based Virtual Machine) on the source host machine.

[0045] Based on this, in this embodiment, the dirty page iteration process (i.e., the control plane operation) in the pre-copy hot migration scheme can be entirely decentralized to the first acceleration device. Therefore, the first acceleration device can locate the source address information of the dirty pages to be transferred in this round based on the dirty page information. In this embodiment, the first acceleration device can traverse the dirty page information, that is, scan the dirty page information bit by bit to find the source address information of the dirty pages. The source address information is used to characterize the location of the dirty page in the source host's memory space. Here, the dirty page iteration process is completely decoupled from the source host; this process has zero dependency on the resources of the source host, meaning that this process does not require any CPU resources from the source host.

[0046] refer to Figure 1a The first acceleration device can send the source address information of the dirty pages to be transferred in this round to the destination host machine.

[0047] In this embodiment, a customized hot migration program can run on the destination host machine. Since there are a large number of idle resources on the destination host machine before the hot migration is completed, this embodiment can use these idle resources to support the operation of the hot migration program. For example, the destination host machine usually reserves CPU resources for the virtual machine to be migrated. In this embodiment, the hot migration program can use the CPU resources allocated for the migrated virtual machine on the destination host machine to support its operation. It should be understood that the CPU resources allocated for the migrated virtual machine are idle and available before the virtual machine migration is completed. Of course, this is only an example. In this embodiment, other idle resources on the destination host machine can also be used to support the operation of the hot migration program. This embodiment does not limit this.

[0048] The destination host machine can cooperate with the first acceleration device based on a customized hot migration program running on it to implement the dirty page transfer process (i.e., data plane operation) in the pre-copy hot migration. Driven by the customized hot migration program, the destination host machine can initiate a dirty page read request based on the obtained source address information. Since this embodiment has changed the traditional architecture where the source host actively pushes dirty pages, the destination host machine can also include the destination address information allocated to the dirty page to be transferred in this round in the dirty page read request, so that the dirty page can be written to the correct location in the destination host machine's memory space after being read back. During the hot migration process, the addresses allocated for memory data on the source host machine and the addresses reserved for memory data on the destination host machine are usually in one-to-one correspondence. Therefore, upon receiving the source address information, the destination host machine can easily and quickly find the destination address corresponding to the dirty page based on this correspondence and include it in the dirty page read request.

[0049] Figure 1b A schematic diagram of another virtual machine hot migration system provided as an exemplary embodiment of this application. (Reference) Figure 1b Preferably, in this embodiment, a second acceleration device may also be configured on the target host side. On this basis, the target host also cooperates with the first and second acceleration devices to implement the dirty page transfer process (i.e., the data plane operation) in the pre-copy hot migration.

[0050] refer to Figure 1b With the addition of a second acceleration device, the first acceleration device can send the source address information of the dirty page to be transferred to the second acceleration device. The second acceleration device can then provide the received source address information to the destination host and trigger the destination host to generate a dirty page read request. The destination host can then send the dirty page read request to its corresponding second acceleration device, as described in the reference. Figure 1bFor the second acceleration device, it can read the dirty pages to be transferred in this round from the source host according to the dirty page read request initiated by the destination host. That is, the first and second acceleration devices can cooperate to respond to the dirty page read request initiated by the destination host. Since the first acceleration device is connected to the source host, it can read the dirty pages to be transferred in this round from the source host and then return them to the second acceleration device. It should be noted that in this embodiment, the reading order of the dirty pages is not limited during the cooperation between the first and second acceleration devices to read the dirty pages to be transferred in this round.

[0051] Preferably, in this embodiment, the first acceleration device can read the dirty pages to be transferred in this round from the source host machine using direct memory access (DMA) and then provide the read dirty pages to be transferred in this round to the second acceleration device. DMA allows the first acceleration device to successfully read the dirty pages to be transferred in this round from the source host machine without relying on the CPU resources of the source host machine. In this way, the dirty page transfer process is completely decoupled from the source host machine, and the process has zero dependency on the resources of the source host machine, that is, the process does not require any CPU resources of the source host machine.

[0052] Thus, in this embodiment, the work of the control plane and data plane during the hot migration process is completely decoupled from the source host machine and executed on the first acceleration device configured for the source host machine. Therefore, the source host machine does not need to perform any tasks related to hot migration and does not need to pay any CPU resources for it, truly achieving zero dependence on the resources of the source host machine.

[0053] Continue to refer to Figure 1a After the first acceleration device reads the dirty pages to be transferred in this round from the source host, it can provide the dirty pages to be transferred in this round to the destination host. Based on Figure 1b In the system architecture shown, in this embodiment, the first acceleration device can send the dirty pages to be transferred in this round back to the second acceleration device; the second acceleration device can write the acquired dirty pages to be transferred in this round into the destination host machine. Preferably, in this embodiment, the second acceleration device can write the dirty pages to be transferred in this round into the destination host machine using Direct Memory Access (DMA). As mentioned above, the dirty page read request initiated by the destination host machine carries the destination address information of the dirty pages on the destination host machine. Therefore, here, the second acceleration device can write the received dirty pages to be transferred in this round into the corresponding address on the destination host machine according to the destination address information in the dirty page read request. In this way, after issuing a dirty page read request, the destination host machine can receive the dirty pages to be transferred in this round written by its corresponding second acceleration device, thereby completing the migration of dirty pages in this round.

[0054] As can be seen, the source host's CPU is completely uninvolved in the entire dirty page transfer process; only the destination host's CPU and the first acceleration device (and a second acceleration device in the preferred scheme) participate in completing the dirty page transfer. Therefore, on the one hand, zero CPU resource consumption is achieved on the source host, and virtual machines running on the source host will not experience service jitter due to CPU resource contention; on the other hand, the performance of dirty page transfer is not limited by the source host's CPU resources. By using the idle CPU on the destination host and the acceleration devices on both sides to move dirty pages, the dirty page transfer performance can be maintained at the hardware's peak, ensuring the performance of dirty page transfer.

[0055] Accordingly, in this embodiment, acceleration devices are respectively located on the source host and the destination host. On the source host side, the acceleration device reads dirty page information from the source host during the hot migration process; based on the dirty page information, it finds the source address information of the dirty page to be transferred in this round; and sends the source address information to the destination host side. On the destination host side, the source address information can be obtained; a dirty page read request is initiated based on the source address information, and the acceleration device on the source host side reads the dirty page to be transferred in this round from the source host. Accordingly, this embodiment provides an optimization scheme for the traditional hot migration architecture. By configuring acceleration devices on the source host side, the hot migration work of the control plane and data plane can be completely offloaded to the acceleration device on the source host side. The hot migration work can be completed by the cooperation of the destination host and the acceleration device, which achieves zero dependence of the hot migration work on the source host resources and effectively improves the efficiency of the hot migration work.

[0056] In the above or the following embodiments, the dirty page transfer process can be implemented by using Remote Direct Memory Access (RDMA).

[0057] In this embodiment, the destination host can generate a Remote Direct Memory Access (RDMA) instruction as a dirty page read request based on the source address information and the destination address information allocated to the dirty pages to be transferred in this round on the destination host. If the destination host is equipped with a second acceleration device, it can send the RDMA instruction to the second acceleration device. Correspondingly, the second acceleration device can read the dirty pages to be transferred in this round from the source host using the first acceleration device in the RDMA manner. Of course, if the destination host is not equipped with a second acceleration device, it can also independently read the dirty pages to be transferred in this round from the source host using the first acceleration device in the RDMA manner. Figure 2 This is a schematic diagram illustrating a hardware implementation of an acceleration device provided for an exemplary embodiment of this application. (Reference) Figure 2In terms of hardware implementation, to support the dirty page transfer process using RDMA, in this embodiment, the first acceleration device may include a first RDMA component, and the second acceleration device may include a second RDMA component. The RDMA component has RDMA communication capabilities. Optionally, in this embodiment, programmable devices such as FPGAs can be used to simulate the RDMA component, RDMA network cards can be used as the RDMA component, or other devices with RDMA capabilities based on ASICs can be used as the RDMA component. This embodiment does not limit the hardware implementation form of the RDMA component. In particular, FPGAs, due to their application-specific integrated circuits, are particularly suitable for the computationally intensive but logically simple working characteristics of the dirty page transfer process in this embodiment. The RDMA component simulated by the FPGA can excellently complete the data transmission and reception work in the dirty page transfer process. Of course, the hardware implementation forms of the RDMA components mentioned here are only exemplary, and this embodiment is not limited to them.

[0058] Based on this, in an exemplary dirty page transfer scheme: for the second RDMA component, the RDMA instructions issued by the destination host for each dirty page to be transferred in this round can be stored in the send queue; each RDMA instruction is encapsulated into an RDMA read message and sent to the receive queue maintained by the first RDMA component; for the first RDMA component, the source address information of the relevant dirty page can be parsed from the RDMA read messages in the receive queue; the relevant dirty page read from the source host according to the source address information is encapsulated into an RDMA acknowledgment message and sent to the second RDMA component, specifically, it can be sent to the acknowledgment queue maintained by the second RDMA component. In this way, the first RDMA component can assist the second RDMA component in reading the dirty pages to be transferred in this round from the source host in the manner of Remote Direct Memory Access (RDMA).

[0059] Optionally, considering the need to ensure the integrity and order of each round of dirty page migration in the pre-copy hot migration scheme, the second RDMA component can also maintain a completion queue in this exemplary scheme. The second RDMA component can maintain the completion status of the dirty pages to be transferred in this round in the completion queue. For the destination host, a completion queue of dirty pages to be transferred in this round can be pre-created in the second RDMA component. After any dirty page in the queue is transferred, the second RDMA component can fill in the completion flag corresponding to the dirty page in the completion queue. In this way, the destination host can perceive the completion status of each dirty page in the completion queue by pulling. After all dirty pages in the completion queue have obtained completion flags, the destination host can confirm that all dirty pages to be transferred in this round have been transferred and can start the next round of migration. It should be understood that the destination host and the second RDMA component can also use other methods to synchronize whether all dirty pages to be transferred in this round have been completed. For example, the second RDMA component can generate a notification event after each dirty page is transferred and provide it to the destination host so that the destination host can sense which dirty pages have been transferred and can independently determine whether all dirty pages to be transferred in this round have been completed, etc. This embodiment does not limit this.

[0060] In this exemplary scheme, the first RDMA component and the second RDMA component can maintain their respective queues and exchange relevant messages according to the RDMA standard protocol to respond to RDMA commands initiated by the destination host in an RDMA manner. Of course, this embodiment is not limited to this; the first RDMA component and the second RDMA component can also respond to RDMA requests initiated by the destination host according to other custom transmission protocols. That is, the interaction details between the first RDMA component and the second RDMA component are not limited to the exemplary scheme described above, and will not be elaborated further here.

[0061] In addition to the RDMA component, the first acceleration device may also include a first dirty page iteration component, and the second acceleration device may also include a second dirty page iteration component. Optionally, in this embodiment, the dirty page iteration component may be implemented using a system-on-a-chip (SoC) or a dedicated host machine. An SoC, also known as a system-on-a-chip, is a product, an integrated circuit with a specific purpose, containing a complete system and all embedded software. Therefore, it is particularly suitable for the complex but computationally low dirty page iteration process in this embodiment. The SoC chip can effectively complete the dirty page information traversal during the dirty page iteration process. When using a dedicated host machine to implement the dirty page iteration component, multiple source hosts can share the same dedicated host machine. This dedicated host machine can provide dirty page iteration function support for different source hosts. For example, the dedicated host machine can run different dirty page iteration processes for different source hosts to isolate the dirty page iteration work corresponding to different source hosts. The working logic of each dirty page iteration process can be referred to the description of the dirty page iteration process in the first acceleration device above, and will not be repeated here.

[0062] Based on this, in this embodiment, the first dirty page iteration component can be responsible for the aforementioned operations during the hot migration process, such as reading dirty page information from the source host, finding the source address information of the dirty page to be transferred in this round based on the dirty page information, and sending the source address information to the second acceleration device; the second dirty page iteration component can be responsible for the operation of providing the source address information to the destination host.

[0063] It is worth noting that when implementing the dirty page iteration component in hardware form such as a SOC chip and a dedicated host machine, the aforementioned RDMA component can be used as a hardware bridge between the dirty page iteration component and the source / destination host machine. In this way, the first dirty page iteration component can use the first RDMA component as a hardware bridge (e.g., an FPGA channel) to read dirty page information from the source host machine, and the second dirty page iteration component and the destination host machine can also use the second RDMA component as a hardware bridge to transfer the address information of the dirty pages to be transferred in this round. Of course, this is only an example, and this embodiment is not limited to it.

[0064] Accordingly, in this embodiment, the acceleration device may include an RDMA component and a dirty page iteration component. The dirty page iteration component can be used for the dirty page iteration process, while the RDMA component can be used for the dirty page transfer process.

[0065] In this embodiment, besides using the aforementioned Remote Direct Memory Access (RDMA) method to implement the dirty page transfer process, other communication methods can also be used. For example, TCP can be used to implement the dirty page transfer process. In TCP mode, the acceleration device can be implemented using a SOC chip or a dedicated host machine. In this example, the acceleration device no longer needs to be configured with an RDMA component. Instead, the SOC chip or dedicated host machine corresponding to the source host machine can directly read dirty page information and the dirty pages to be transferred in this round from the source host machine through DMA or other methods. The acceleration devices at both ends can interact through the TCP protocol, and the SOC chip or dedicated host machine connected to the destination host machine can directly receive dirty page read requests from the destination host machine or write the dirty pages to be transferred in this round to the destination host machine through DMA or other methods. This embodiment does not limit the communication method used in the dirty page transfer process. For different communication methods, appropriate hardware can be used to implement the acceleration device.

[0066] Therefore, in this embodiment, communication methods such as RDMA can be used to implement the dirty page transfer process during virtual machine hot migration. Throughout the process, the source host does not need to participate, ensuring zero dependency on the source host. Furthermore, the dirty page transfer process can be completed unilaterally on the destination host, achieving a post-copy-like pre-copy hot migration effect. That is, the work that would normally be performed on the source host is instead performed on the destination host. This achieves zero resource consumption on the source host, eliminating virtual machine jitter caused by hot migration and freeing hot migration performance from the constraints of CPU resources on the source host, thus maintaining stable high performance during hot migration.

[0067] Figure 3 This is a flowchart illustrating a method for hot migration of a virtual machine, provided as another exemplary embodiment of this application. The method can be executed by an acceleration device, which can be implemented as a combination of software and / or hardware, and can be integrated into a first acceleration device configured for the source host. (Reference) Figure 3 The method includes:

[0068] Step 300: During the hot migration process, read dirty page information from the source host machine;

[0069] Step 301: Based on the dirty page information, find the source address information of the dirty pages to be transferred in this round;

[0070] Step 302: Send the source address information to the destination host machine so that the destination host machine can initiate a dirty page read request based on the source address information;

[0071] Step 303: Read the dirty pages to be transferred in this round from the source host machine;

[0072] Step 304: Send the read dirty pages to be transferred in this round to the destination host machine in response to the dirty page read request.

[0073] In an optional embodiment, the target host machine is configured with a second acceleration device, and step 302 includes:

[0074] The source address information is sent to the second acceleration device configured for the destination host, so that the second acceleration device can trigger the destination host to initiate a dirty page read request based on the source address information;

[0075] Step 304 includes: sending the read dirty pages to be transferred in this round to the second acceleration device, so that the second acceleration device can respond to the dirty page read request.

[0076] In an alternative embodiment, step 303 may include:

[0077] The dirty pages to be transferred in this round are read from the source host machine using the Direct Memory Access (DMA) method.

[0078] In an optional embodiment, the destination host can generate a Remote Direct Memory Access (RDMA) instruction as a dirty page read request based on the source address information and the destination address information allocated to the dirty pages to be transferred in this round on the destination host; and send the RDMA instruction to the second acceleration device.

[0079] The step of sending the dirty pages to be transferred in the current round to the second acceleration device may include: sending the dirty pages to be transferred in the current round to the second acceleration device in the manner of Remote Direct Memory Access (RDMA).

[0080] In one optional embodiment, the first acceleration device includes a first RDMA component, and the second acceleration device includes a second RDMA component; the method specifically includes:

[0081] The RDMA read message sent by the second RDMA component is stored in the receive queue. The RDMA read message is generated by the second RDMA component storing the RDMA instructions issued by the destination host for each dirty page to be transmitted in this round into the send queue and encapsulating each RDMA instruction.

[0082] Parse the source address information of the relevant dirty pages from the RDMA read messages in the receive queue;

[0083] The dirty pages read from the source host based on the source address information are encapsulated into an RDMA response message and sent to the second RDMA component to assist the second RDMA component in reading the dirty pages to be transferred in this round from the source host in the manner of Remote Direct Memory Access (RDMA).

[0084] In an alternative embodiment, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities.

[0085] In an optional embodiment, the first acceleration device further includes a first dirty page iteration component, and the second acceleration device further includes a second dirty page iteration component, with the first dirty page iteration component performing steps 300-302.

[0086] In one alternative embodiment, the first dirty page iteration component and the second dirty page iteration component are implemented using a system-on-a-chip (SoC) or a dedicated host machine.

[0087] In one optional embodiment, the first acceleration device and the second acceleration device are implemented using a SOC chip or a dedicated host machine, and the first acceleration device and the second acceleration device read the dirty pages to be transferred in this round from the source host machine according to the TCP protocol.

[0088] It is worth noting that the technical details of the above embodiments of the virtual machine migration method can be found in the description of the first acceleration device in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0089] Figure 4 This is a flowchart illustrating another virtual machine hot migration method provided as an exemplary embodiment of this application. The method can be executed by an acceleration device, which can be implemented as a combination of software and / or hardware, and can be integrated into a second acceleration device configured for the target host. (See reference...) Figure 4 The method includes:

[0090] Step 400: During the hot migration process, receive the source address information of the dirty pages to be transferred in this round provided by the first acceleration device configured for the source host.

[0091] Step 401: Provide the source address information to the destination host machine;

[0092] Step 402: According to the dirty page read request initiated by the destination host based on the source address information, use the first acceleration device to read the dirty pages to be transferred in this round from the source host;

[0093] Step 403: Write the obtained dirty pages to be transferred in this round into the destination host machine.

[0094] In an optional embodiment, step 403 includes: writing the dirty pages to be transferred in this round into the destination host machine in the manner of direct memory access (DMA).

[0095] In an optional embodiment, the destination host machine may generate a Remote Direct Memory Access (RDMA) instruction as a dirty page read request based on the source address information and the destination address information allocated to the dirty pages to be transferred in this round on the destination host machine; and send the RDMA instruction to the second acceleration device; step 402 may include:

[0096] Using the Remote Direct Memory Access (RDMA) method, the dirty pages to be transferred in this round are read from the source host using the first acceleration device.

[0097] In one optional embodiment, the first acceleration device includes a first RDMA component, and the second acceleration device includes a second RDMA component; the method specifically includes:

[0098] The RDMA instructions issued by the destination host for each dirty page to be transferred in this round are stored in the transmit queue; each RDMA instruction is encapsulated into an RDMA read message and sent to the receive queue maintained in the first RDMA component, so that the first RDMA component can parse the source address information of the relevant dirty page from the RDMA read message in the receive queue; the relevant dirty page read from the source host according to the source address information is encapsulated into an RDMA acknowledgment message and sent to the second RDMA component.

[0099] In an optional embodiment, the method further includes: maintaining the completion status of the dirty pages to be transferred in the current round in the completion queue, so that the destination host can determine that all the dirty pages to be transferred in the current round have been migrated after the completion status of the dirty pages to be transferred in the current round in the completion queue, and then start the next round of migration.

[0100] In an alternative embodiment, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities.

[0101] In an optional embodiment, the first acceleration device further includes a first dirty page iteration component, and the second acceleration device further includes a second dirty page iteration component, wherein the second dirty page iteration component is responsible for providing the source address information to the destination host machine.

[0102] In one alternative embodiment, the first dirty page iteration component and the second dirty page iteration component are implemented using a system-on-a-chip (SoC) or a dedicated host machine.

[0103] In one optional embodiment, the first acceleration device and the second acceleration device are implemented using a SOC chip or a dedicated host machine, and the first acceleration device and the second acceleration device read the dirty pages to be transferred in this round from the source host machine according to the TCP protocol.

[0104] It is worth noting that the technical details of the above embodiments of the virtual machine migration method can be found in the description of the second acceleration device in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0105] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 801, 802, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different devices, components, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0106] Figure 5 This is a schematic diagram of an acceleration device provided as another exemplary embodiment of this application. The acceleration device can be connected to a source host machine. Figure 5 As shown, the acceleration device includes: a dirty page iteration component 51 and a communication component 52;

[0107] The dirty page iteration component 51 is used to read dirty page information from the source host during hot migration; find the source address information of the dirty page to be transferred in this round based on the dirty page information; and send the source address information to the destination host so that the destination host can initiate a dirty page read request based on the source address information.

[0108] Communication component 52 is used to read the dirty pages to be transferred in the current round from the source host and send the read dirty pages to be transferred in the current round to the destination host in response to the dirty page read request.

[0109] In an optional embodiment, the destination host machine is configured with a second acceleration device, and the dirty page iteration component 51, during the process of sending source address information to the destination host machine, can be used to:

[0110] The source address information is sent to the second acceleration device configured for the destination host, so that the second acceleration device can trigger the destination host to initiate a dirty page read request based on the source address information;

[0111] In the process of sending the dirty pages to be transferred in the current round to the destination host, the dirty page iteration component 51 can be used to: send the dirty pages to be transferred in the current round to the second acceleration device, so as to cooperate with the second acceleration device to respond to the dirty page read request.

[0112] In an optional embodiment, during the process of reading dirty pages to be transferred in the current round from the source host, the communication component 52 can be used for:

[0113] The dirty pages to be transferred in this round are read from the source host machine using the Direct Memory Access (DMA) method.

[0114] In an optional embodiment, the destination host can generate a Remote Direct Memory Access (RDMA) instruction as a dirty page read request based on the source address information and the destination address information allocated to the dirty pages to be transferred in this round on the destination host; and send the RDMA instruction to the second acceleration device.

[0115] In the process of sending the dirty pages to be transferred in the current round to the second acceleration device, the communication component 52 can be used to: send the dirty pages to be transferred in the current round to the second acceleration device in the manner of Remote Direct Memory Access (RDMA).

[0116] In an optional embodiment, the communication component 52 may include a first RDMA component, and the second acceleration device may include a second RDMA component; the dirty page iteration component 51 may be used for:

[0117] The RDMA read message sent by the second RDMA component is stored in the receive queue. The RDMA read message is generated by the second RDMA component storing the RDMA instructions issued by the destination host for each dirty page to be transmitted in this round into the send queue and encapsulating each RDMA instruction.

[0118] Parse the source address information of the relevant dirty pages from the RDMA read messages in the receive queue;

[0119] The dirty pages read from the source host based on the source address information are encapsulated into an RDMA response message and sent to the second RDMA component to assist the second RDMA component in reading the dirty pages to be transferred in this round from the source host in the manner of Remote Direct Memory Access (RDMA).

[0120] In an alternative embodiment, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities.

[0121] In an optional embodiment, the dirty page iteration component in this acceleration device and the dirty page iteration component in the second acceleration device connected to the target host are implemented using a system-on-a-chip (SoC) or a dedicated host.

[0122] In one optional embodiment, the acceleration device and the second acceleration device are implemented using a SOC chip or a dedicated host machine, and the acceleration device and the second acceleration device read the dirty pages to be transferred in this round from the source host machine according to the TCP protocol.

[0123] Furthermore, the acceleration device also includes other components such as a power supply component 53.

[0124] It is worth noting that the technical details of the various embodiments of the acceleration device mentioned above can be found in the relevant description of the first acceleration device in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0125] Figure 6 This is a schematic diagram of another acceleration device provided as a further exemplary embodiment of this application. This acceleration device can be connected to a target host machine. Figure 6 As shown, the acceleration device includes: a dirty page iteration component 61 and a communication component 62;

[0126] The dirty page iteration component 61 is used to receive, during the hot migration process, the source address information of the dirty pages to be transferred in this round provided by the first acceleration device configured for the source host; and to provide the source address information to the destination host.

[0127] The communication component 62 is used to read the dirty pages to be transferred in this round from the source host according to the dirty page read request initiated by the destination host based on the source address information, using the first acceleration device; and to write the obtained dirty pages to be transferred in this round into the destination host.

[0128] In an optional embodiment, during the process of the communication component 62 writing the dirty pages to be transferred in the current round to the destination host machine, it can be used to: write the dirty pages to be transferred in the current round to the destination host machine in the manner of direct memory access (DMA).

[0129] In an optional embodiment, the destination host can generate a Remote Direct Memory Access (RDMA) instruction as a dirty page read request based on the source address information and the destination address information allocated to the dirty pages to be transferred in this round on the destination host; and send the RDMA instruction to the acceleration device; the communication component 62 can be used to:

[0130] Using the Remote Direct Memory Access (RDMA) method, the dirty pages to be transferred in this round are read from the source host using the first acceleration device.

[0131] In an optional embodiment, the first acceleration device includes a first RDMA component, and the communication component 62 may include a second RDMA component; the method specifically includes:

[0132] The RDMA instructions issued by the destination host for each dirty page to be transferred in this round are stored in the transmit queue; each RDMA instruction is encapsulated into an RDMA read message and sent to the receive queue maintained in the first RDMA component, so that the first RDMA component can parse the source address information of the relevant dirty page from the RDMA read message in the receive queue; the relevant dirty page read from the source host according to the source address information is encapsulated into an RDMA acknowledgment message and sent to the second RDMA component.

[0133] In an optional embodiment, the dirty page iteration component 61 can also be used to: maintain the completion status of the dirty pages to be transferred in the current round in the completion queue, so that the destination host can determine that all the dirty pages to be transferred in the current round have been migrated after the completion status of the dirty pages to be transferred in the current round in the completion queue, and then start the next round of migration.

[0134] In an alternative embodiment, the first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities.

[0135] In an optional embodiment, the first dirty page iteration component in the first acceleration device on the source host and the dirty page iteration component 61 in this acceleration side are implemented using a system-on-a-chip (SoC) or a dedicated host.

[0136] In one optional embodiment, the first acceleration device and the local acceleration device are implemented using a SOC chip or a dedicated host machine, and the first acceleration device and the local acceleration device read the dirty pages to be transferred in this round from the source host machine according to the TCP protocol.

[0137] Furthermore, the acceleration device also includes other components such as a power supply component 63.

[0138] It is worth noting that the technical details of the various embodiments of the acceleration device mentioned above can be found in the description of the second acceleration device in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0139] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by the first acceleration device or the second acceleration device in the above method embodiments.

[0140] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0144] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0145] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0146] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0147] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0148] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A virtual machine hot migration system, characterized in that, include: The source host machine and its corresponding first acceleration device, and the destination host machine; The first acceleration device is used to read dirty page information from the source host during the hot migration process; Based on the dirty page information, locate the source address information of the dirty pages to be transferred in this round; Send the source address information to the destination host machine; The destination host machine is used to obtain the source address information; A dirty page read request is initiated based on the source address information; the first acceleration device is used to obtain the dirty pages to be transferred in this round from the source host.

2. The system according to claim 1, characterized in that, The destination host machine corresponds to a second acceleration device. During the process of sending the source address information to the destination host machine, the first acceleration device is used for: The source address information is sent to the second acceleration device configured for the destination host machine; The second acceleration device is used for: Provide the source address information to the destination host machine; According to the dirty page read request, the first acceleration device is used to read the dirty pages to be transferred in this round from the source host. The acquired dirty pages to be transferred in this round are written into the target host machine.

3. The system according to claim 2, characterized in that, The first acceleration device is further configured to: read the dirty pages to be transferred in the current round from the source host in the manner of direct memory access (DMA); and provide the read dirty pages to be transferred in the current round to the second acceleration device; During the process of writing the acquired dirty pages to be transferred in this round into the target host machine, the second acceleration device is used to: write the dirty pages to be transferred in this round into the target host machine in the manner of direct memory access (DMA).

4. The system according to claim 2, characterized in that, The destination host is used to: generate a Remote Direct Memory Access (RDMA) instruction as a dirty page read request based on the source address information and the destination address information allocated on the destination host for the dirty page to be transferred in this round; Send the remote direct memory access command to the second acceleration device; In the process of reading dirty pages to be transferred in the current round from the source host using the first acceleration device, the second acceleration device is used to: read dirty pages to be transferred in the current round from the source host using the first acceleration device in the manner of Remote Direct Memory Access (RDMA).

5. The system according to claim 4, characterized in that, The first acceleration device includes a first RDMA component, and the second acceleration device includes a second RDMA component; The second RDMA component is used to store the RDMA instructions issued by the destination host for each dirty page to be transferred in this round into the transmission queue; Each RDMA instruction is encapsulated into an RDMA read message and then sent to the receive queue maintained in the first RDMA component. The first RDMA component is used to parse the source address information of the relevant dirty page from the RDMA read message in the receive queue; The relevant dirty pages read from the source host based on the source address information are encapsulated into an RDMA response message and sent to the second RDMA component to assist the second RDMA component in reading the dirty pages to be transferred in this round from the source host in the manner of Remote Direct Memory Access (RDMA).

6. The system according to claim 5, characterized in that, The second RDMA component is also used to maintain the completion status of the dirty pages to be transferred in the current round in the completion queue; The destination host machine is further configured to determine, based on the completion status of the dirty pages to be transferred in the current round in the completion queue, that all dirty pages to be transferred in the current round have been migrated, and then initiate the next round of migration.

7. The system according to claim 5, characterized in that, The first RDMA component and the second RDMA component are implemented using an FPGA, an RDMA network card, or other ASIC-based devices with RDMA capabilities.

8. The system according to claim 5, characterized in that, The first acceleration device further includes a first dirty page iteration component, and the second acceleration device further includes a second dirty page iteration component. The first dirty page iteration component is responsible for reading dirty page information from the source host during the hot migration process, finding the source address information of the dirty page to be transferred in this round based on the dirty page information, and sending the source address information to the second acceleration device. The operation of providing the source address information to the destination host machine is undertaken by the second dirty page iteration component.

9. The system according to claim 8, characterized in that, The first dirty page iteration component and the second dirty page iteration component are implemented using a system-on-a-chip (SoC) or a dedicated host machine.

10. The system according to claim 1, characterized in that, The destination host machine runs a hot migration program, which uses the CPU resources allocated on the destination host machine for the migrated virtual machine to perform the operation of initiating a dirty page read request based on the source address information.

11. The system according to claim 2, characterized in that, The first acceleration device and the second acceleration device are implemented using a SOC chip or a dedicated host machine. The first acceleration device and the second acceleration device read the dirty pages to be transferred in this round from the source host machine according to the TCP protocol.

12. A method for hot migration of virtual machines, characterized in that, The method, applicable to the first acceleration device corresponding to the source host machine, includes: During the hot migration process, dirty page information is read from the source host machine; Based on the dirty page information, locate the source address information of the dirty pages to be transferred in this round; The source address information is sent to the destination host machine so that the destination host machine can initiate a dirty page read request based on the source address information; Read the dirty pages to be transferred in this round from the source host machine; The dirty pages to be transferred in this round are provided to the destination host in response to the dirty page read request.

13. A method for hot migration of virtual machines, characterized in that, The method, applicable to a second acceleration device configured for a target host machine, includes: During hot migration, the source address information of the dirty pages to be transferred in this round is received from the first acceleration device configured for the source host. Provide the source address information to the destination host machine; According to the dirty page read request initiated by the destination host based on the source address information, the first acceleration device is used to read the dirty pages to be transferred in this round from the source host; The acquired dirty pages to be transferred in this round are written into the target host machine.

14. An acceleration device, characterized in that, Connects to the source host machine, including dirty page iteration components and communication components; The dirty page iteration component is used to read dirty page information from the source host machine through the communication component during hot migration. Based on the dirty page information, locate the source address information of the dirty pages to be transferred in this round; The source address information is sent to the destination host machine so that the destination host machine can initiate a dirty page read request based on the source address information; The communication component is used to read the dirty pages to be transferred in this round from the source host machine; The dirty pages to be transferred in this round are sent to the destination host in response to the dirty page read request.

15. An acceleration device, characterized in that, Connects to the target source host machine, including dirty page iteration components and communication components; The dirty page iteration component is used to receive, during the hot migration process, the source address information of the dirty pages to be transferred in this round provided by the first acceleration device configured for the source host. Provide the source address information to the destination host machine; The communication component is used to read the dirty pages to be transmitted in this round from the source host according to the dirty page read request initiated by the destination host based on the source address information, using the first acceleration device; The acquired dirty pages to be transferred in this round are written into the target host machine.

16. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by one or more processors, the one or more processors perform the live migration method for a virtual machine as described in any one of claims 12 or 13.

Images