Virtual machine migration method, and device, computer program product and storage medium

Abstract

Provided in the embodiments of the present disclosure are a virtual machine migration method, and a device, a computer program product and a storage medium. For a target virtual machine that uses a hardware GPU to perform rendering acceleration, after receiving a live migration trigger instruction, a source host may, in addition to starting the execution of a conventional live migration process, further control the target virtual machine to stop sending instructions to a virtual GPU in use, wait for the completion of the execution of each rendering instruction that is being executed, and then start to migrate, to a destination host, device state data corresponding to the virtual GPU as well as rendering data and state data that are generated by a hardware rendering process corresponding to the virtual GPU, so as to support, in the destination host, state restoration of the virtual GPU and the hardware rendering process that are involved in the target virtual machine. Accordingly, it is no longer necessary to replay instructions on the destination host, thereby effectively improving the migration efficiency of such virtual machines.

Description

[0001] The application discloses a virtual machine migration method, device, computer program product and storage medium, and relates to the technical field of cloud computing, in particular to a virtual machine migration method, device, computer program product and storage medium. Background art Virtual machine live migration refers to the process of migrating a running virtual machine from a source host to a destination host. At present, based on technologies such as input / output virtualization (Virtual I / O GPU, Virtio-GPU) for a graphics processing unit (GPU), a hardware GPU can be used in a virtual machine to process a part of complex rendering work, thereby realizing rendering acceleration. For such a virtual machine, an instruction playback mode is usually adopted to ensure that all subjects related to rendering acceleration can be restored to a pre-migration state. However, this mode needs to re-run all historical rendering instructions of the virtual machine on the source host after starting the virtual machine on the destination host, which will cause a long playback time consumption, thereby causing a long migration time for the virtual machine. Aspects of the application provide a virtual machine migration method, device, computer program product and storage medium, to improve the migration efficiency for a virtual machine using a hardware GPU for rendering acceleration. An embodiment of the application provides a virtual machine migration method applicable to a source host where a target virtual machine is located, and the method comprises the following steps: in response to a live migration trigger instruction for the target virtual machine, controlling the target virtual machine to stop sending rendering instructions to a virtual GPU used by the target virtual machine; during the live migration of the target virtual machine, if it is monitored that the rendering instruction execution of the virtual GPU received is completed, device state data corresponding to the virtual GPU is migrated to a destination host, so as to support the destination host to restore the state of the virtual GPU; and rendering data and state data generated by a hardware rendering process corresponding to the virtual GPU are migrated to the destination host, so as to support the destination host to restore the state of the hardware rendering process. An embodiment of the application further provides a virtual machine migration method applicable to a destination host to which a target virtual machine is migrated, and the method comprises the following steps: in response to a recovery instruction for the target virtual machine, initializing the target virtual machine; after the initialization is completed, restoring the state of a virtual GPU required to be used by the target virtual machine according to device state data corresponding to the virtual GPU migrated from a source host of the target virtual machine; and restoring the state of a hardware rendering process according to rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU migrated from the source host.The embodiment of the present disclosure further provides a computing device, comprising a memory, a processor and a communication component; the memory is configured to store one or more computer instructions; the processor is coupled with the memory and the communication component, and is configured to execute the one or more computer instructions for performing the foregoing virtual machine migration method as a source host or a destination host. The embodiment of the present disclosure further provides a computer readable storage medium storing a computer program, which causes one or more processors to execute the foregoing virtual machine migration method when the computer program is executed by the one or more processors. The embodiment of the present disclosure further provides a computer program product, comprising a computer program, which causes one or more processors to execute the foregoing virtual machine migration method when the computer program is executed by the one or more processors. In the embodiment of the present disclosure, a virtual machine migration method is proposed for a virtual machine using hardware GPU for rendering acceleration. After receiving a hot migration trigger instruction, the source host can control the target virtual machine to stop sending instructions to the virtual GPU used by the target virtual machine, and after the execution of each rendering instruction in progress is completed, the source host can start migrating the device state data corresponding to the virtual GPU used by the target virtual machine and the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU to the destination host, so as to support the state recovery of the virtual GPU and the hardware rendering process involved in the target virtual machine in the destination host. Accordingly, in the embodiment of the present disclosure, the instruction playback is no longer required on the destination host, but the detailed migration of the above several aspects of data is completed in the original hot migration process of the virtual machine, so that the subjects (such as the virtual GPU and the hardware rendering process) related to rendering acceleration can be restored to the state before migration on the destination host side, which can effectively improve the migration efficiency for such virtual machines. BRIEF DESCRIPTION OF DRAWINGS The accompanying drawings used to provide a further understanding of the present disclosure, constitute a part of the present disclosure, the illustrative embodiments of the present disclosure and the description thereof serve to explain the present disclosure, and do not constitute an improper limitation on the present disclosure.In the drawings: Fig. 1 is a flowchart of a virtual machine migration method according to an example embodiment of the present disclosure; Fig. 2 is a schematic diagram of a principle of a target virtual machine using a hardware GPU to perform rendering according to an example embodiment of the present disclosure; Fig. 3 is a flowchart of another virtual machine migration method according to an example embodiment of the present disclosure; Fig. 4 is a schematic diagram of a rendering acceleration principle based on VirtIO technology and Virglrenderer technology according to an embodiment of the present disclosure; Fig. 5 is a flowchart of another virtual machine migration method according to an example embodiment of the present disclosure; Fig. 6 is a flowchart of a virtual machine migration method according to another example embodiment of the present disclosure; Fig. 7 is a flowchart of another virtual machine migration method according to another example embodiment of the present disclosure; and Fig. 8 is a schematic diagram of a computing device according to yet another example embodiment of the present disclosure. DETAILED DESCRIPTION In order to make the objects, technical solutions, and advantages of the present disclosure clearer, the technical solutions of the present disclosure will be described clearly and completely below in connection with the specific embodiments of the present disclosure and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, but not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present disclosure. Before the technical solutions provided by the embodiments of the present disclosure are described in detail, several technical concepts involved in the present disclosure are explained simply as follows. Virtual machine live migration: is a process of migrating a running virtual machine from a source host to a destination host. Virtual machine live migration mainly involves migration of memory data, and can be roughly divided into two migration mechanisms, pre-copy and post-copy. During the live migration process, the downtime of the virtual machine is very short, and therefore, the user is usually not aware of the live migration of the virtual machine.

[0002] VirtIO technology, full name Virtual I / O, is an I / O semi-virtualization technology, which adopts front-end and back-end architecture, including device driver, virtualization device, and virtualization queue (Vring) for communication between device driver and virtualization device. The device driver is located in the virtual machine (as a front-end driver) and is the corresponding driver of the virtualization device in the virtual machine; the virtualization device is generated by the virtualization of the external device by the virtual machine manager (VMM) (as a back-end device). The device driver and the virtualization device can read and write data from the virtualization queue. The device driver can save the I / O instruction to the virtualization queue, and then notify the virtualization device, and the virtualization device can obtain the I / O instruction from the virtualization queue for processing.

[0003] VirtlO-GPU, full name Virtual I / O GPU, is a GPU virtualization scheme based on the VirtIO technology. It can be understood that a virtual GPU (as a backend device) is generated by virtualizing a hardware GPU based on the VirtIO technology, and a device driver (as a front-end driver) is created for the virtual GPU in the virtual machine. The application in the virtual machine can send rendering instructions to the device driver, and the device driver can pass the rendering instructions to the virtual GPU, which can call the hardware GPU to perform rendering. Software rendering refers to using a CPU to perform rendering. For a virtual machine, it can be understood as using a CPU for rendering acceleration. Hardware rendering refers to using a hardware GPU to perform rendering. For a virtual machine, it can be understood as using a hardware GPU for rendering acceleration. Accordingly, the hardware rendering process is a rendering process based on the hardware GPU. As introduced in the background art, at present, if a virtual machine needs to use a hardware GPU for rendering acceleration, after migration of the virtual machine, all historical rendering instructions generated by the virtual machine on the source host need to be re-run on the destination host to ensure that each subject related to rendering can be restored to the state before migration. Therefore, a long playback time is required, which leads to a long migration time for such a virtual machine. To this end, the present disclosure proposes a virtual machine migration scheme based on a state migration mechanism for such a virtual machine, which does not need to perform instruction playback on the destination host, and can ensure that each subject related to rendering acceleration can be restored to the state before migration, thereby improving the migration efficiency of such a virtual machine. The technical solutions provided by the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. FIG. 1 is a flowchart of a virtual machine migration method provided by an example embodiment of the present disclosure. As shown in FIG. 1, the method can be performed by a source host where a target virtual machine to be migrated is located, and can be performed by a virtual machine migration device integrated in the source host. The virtual machine migration device can be implemented as software, hardware, or a combination of software and hardware. In this embodiment, the target virtual machine can be any virtual machine that needs to use a hardware GPU for rendering acceleration. It should be understood that in this embodiment, the source host and the destination host corresponding to the target virtual machine are both equipped with a hardware GPU, and both can support the implementation of the VirtlO-GPU and other technologies that can virtualize the hardware GPU. FIG. 2 is a schematic diagram of a principle of using a hardware GPU for rendering by a target virtual machine.Referring to FIG. 2, the host (source host or destination host) for carrying the target virtual machine is equipped with a hardware GPU, which in this embodiment can be virtualized to generate a virtual GPU available for the target virtual machine. The inventor has found in the course of research that the target virtual machine can send rendering instructions to the virtual GPU used by the target virtual machine during operation, and the virtual GPU can drive the corresponding hardware GPU to execute the rendering instructions through the hardware rendering process mentioned above in the host. On this basis, this embodiment proposes that, referring to FIG. 1, the virtual machine migration method performed by the source host where the target virtual machine is located can include: step 100, in response to a hot migration trigger instruction for the target virtual machine, controlling the target virtual machine to stop sending rendering instructions to the virtual GPU used by the target virtual machine; step 101, during the hot migration of the target virtual machine, if it is monitored that the execution of the rendering instructions received by the virtual GPU is completed, migrating the device state data corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the virtual GPU; and step 102, migrating the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the hardware rendering process. Referring to FIG. 1, in step 100, the source host can start the hot migration process after receiving the hot migration trigger instruction for the target virtual machine. In this embodiment, it can be understood that the virtual machine hot migration technology is logically modified, and the migration logic provided in steps 100-102 in this embodiment is added to the existing migration logic in the virtual machine hot migration technology. The existing migration logic in the virtual machine hot migration technology can include migration logic for memory data and virtual machine device state, etc. Therefore, it can be understood that in this embodiment, one triggering effect of the hot migration trigger instruction is to trigger the source host to start the existing migration logic in the virtual machine hot migration technology for the target virtual machine. Referring to FIG. 1, in step 100, another triggering effect of the hot migration trigger instruction is designed, that is, to trigger the source host to control the target virtual machine to stop sending rendering instructions to the virtual GPU used by the target virtual machine. In other words, after the hot migration trigger instruction for the target virtual machine occurs, the target virtual machine will stop sending rendering instructions to the virtual GPU used by the target virtual machine through the control of the source host, and accordingly, the virtual GPU used by the target virtual machine will stop receiving new rendering instructions. In this embodiment, when the new rendering instructions are stopped, the rendering instructions received by the virtual GPU before this time can not have been executed, which causes the device state data of the virtual GPU to still be changing.To this end, the embodiment proposes that the rendering instructions received by the virtual GPU can be waited to be executed. The time length required by the waiting process is uncertain, which depends on the execution complexity of the rendering instructions and the like. However, the waiting process does not affect the execution of the live migration process of the target virtual machine. For example, during the waiting process, the memory data migration of the target virtual machine is normally performed. Referring to FIG. 1, in step 101, the source host can monitor whether the rendering instructions received by the virtual GPU are executed during the live migration process of the target virtual machine. In actual application, after the rendering instructions are executed, the virtual GPU usually receives a completion notification. Based on this, in the embodiment, the source host can initiate a query operation of the instruction execution state to the virtual GPU, so as to determine whether the rendering instructions received by the virtual GPU have been executed completely. Of course, the virtual GPU can also actively send a completion notification to the source host after confirming that the rendering instructions received by the virtual GPU have been executed completely, so that the source host knows that the rendering instructions received by the virtual GPU have been executed completely. Wherein, after the rendering instructions received by the virtual GPU are executed completely, the device state of the virtual GPU will no longer change, and the hardware rendering process corresponding to the virtual GPU in the source host is still running, but since there is no rendering instruction to be executed, the hardware rendering process will also no longer change. Therefore, in steps 101 and 102 in the embodiment, it is proposed that after it is monitored that the rendering instructions received by the virtual GPU are executed, the state migration of each subject related to rendering acceleration can be started. In the embodiment, each subject related to rendering acceleration at least includes the virtual GPU used by the target virtual machine and the hardware rendering process corresponding to the virtual GPU in the source host. Referring to FIG. 1, in step 101, the source host can migrate the device state data corresponding to the virtual GPU to the destination host, so as to support the destination host to restore the state of the virtual GPU. The inventors find in the research process that in the current virtualization technology of the hardware CPU, the virtual machine manager VMM in the host (the source host or the destination host) is usually used to implement, and the virtual machine manager VMM usually maintains the device state data of the virtual GPU. The virtual machine manager VMM can also be called Hypervisor, which is a software layer running on the host to provide resource isolation, monitoring, management and the like for virtualization. Common Hypervisors include Qemu, VirtualBox and the like. The device state data can describe the working state of the virtual GPU, which can include some static attribute information of the virtual GPU and the values of various state variables involved, and is not limited herein.In the design concept of the embodiment, the migration of the device state data is expected to support the destination host to correctly restore the state of the virtual GPU, and therefore, in actual application, the virtual GPU can be restored to the state before migration can be taken as a goal to set the data items required to be contained in the device state data in step 101. In this way, in step 101, the source host can collect these required data items and migrate to the destination host. Based on this, the device state data maintained for the virtual GPU required to be migrated can be read from the virtual machine manager VMM (which can be understood as the physical memory used by the VMM in the source host in essence) and migrated to the destination host. It is worth mentioning that in the embodiment, the source host can save the device state data corresponding to the virtual GPU to a storage space outside the destination host, and the destination host can read the required device state data from the storage space when state restoration is required, to complete the migration of the device state data. Of course, the source host can also directly send the device state data corresponding to the virtual GPU to the destination host to complete the migration of the device state data, which is not limited in the embodiment. As mentioned before, after the execution of the rendering instruction received by the virtual GPU is completed, the hardware rendering process corresponding to the virtual GPU in the source host will also no longer change. Based on this, in step 102, it is proposed that the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU can be migrated to the destination host to support the destination host to restore the state of the hardware rendering process. The rendering data refers to the data required to be used or generated by the hardware GPU when running the hardware rendering process, which can include but is not limited to pixel data and coordinate data, etc., and no more examples are given here. The state data is used to describe the running state of the hardware rendering process, which includes but is not limited to the identification information of the rendering resources used, whether to use light calculation, and the light source turned on, etc., and no more examples of the data items in the state data are given here. Based on this, in step 102, the source host can be designed with a finding logic for the rendering data and state data generated by the hardware rendering process, so that the source host can find the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU in detail and migrate to the destination host. The inventors have found in the research process that the hardware rendering process is run based on the hardware GPU, but it is usually the source host that is responsible for managing and scheduling the hardware rendering process, and therefore, in the embodiment, based on the finding logic designed for the source host, the source host can comprehensively find the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU and migrate to the destination host.It is worth mentioning that, with reference to FIG. 2, in the embodiment, the technical concept that the source host migrates the state of the virtual GPU and the hardware rendering process related to rendering acceleration is described, which can provide a comprehensive and accurate data basis for the destination host to restore the state of the virtual GPU and the hardware rendering process after the target virtual machine is migrated to the destination host, so as to ensure that the target host can restore the virtual GPU and the hardware rendering process to the state before migration. Of course, in actual application, the main body related to rendering acceleration may not only have the virtual GPU and the hardware rendering process, but also have other main bodies, such as the target virtual machine itself, the device driver of the virtual GPU, etc. However, as mentioned in the foregoing in the embodiment, in the process of hot migration, the source host not only executes the technical logic of state migration provided in the embodiment, but also executes the existing migration logic in the hot migration technology, which includes the migration logic of the memory data of the virtual machine. Therefore, the memory data corresponding to the target virtual machine can be comprehensively migrated to the destination host, and the data related to the state of the other main bodies mentioned above is basically included in the memory data of the target virtual machine. In this way, the memory data migrated to the destination host can provide a comprehensive and accurate data basis for the destination host to restore the state of the other main bodies, so as to ensure that the target host can also restore the other main bodies related to rendering acceleration to the state before migration. Accordingly, the GPU virtualization technology used in the source host in the technical concept provided in the embodiment is not limited. For example, the VirtIO technology mentioned above can be supported for GPU virtualization, and for another example, software simulation technology, direct transmission technology or direct shared technology SR-IOV can be used for GPU virtualization. Under different GPU virtualization technologies, the virtual GPU for the target virtual machine is generated, but the name of the virtual GPU may be different in different GPU virtualization technologies. For example, in the VirtIO technology, the virtual GPU is usually called VirtIO-GPU; for another example, in the SR-IOV technology, the virtual GPU is usually called vGPU. In the technical concept provided in the embodiment, the source host can migrate the state of the virtual GPU generated by any GPU virtualization technology. In addition, in the technical concept provided in the embodiment, the driving technology framework for supporting the virtual GPU driving hardware GPU is also not limited. Regardless of which driving technology framework, the hardware rendering process essentially needs to be run based on the hardware GPU, but there may be differences in the management and scheduling details of the hardware rendering process.In this embodiment, the hardware rendering process is taken as the main body to perform state migration. This underlying and basic perspective can ensure that the technical concept provided by this embodiment can adapt to various driving technology frameworks. For different driving technology frameworks, technical logic adaptability design can be performed in engineering implementation. Preferably, in this embodiment, virglrenderer can be used as the driving technology framework. Virglrenderer is a driving technology framework for delivering OpenGL rendering instructions to a hardware GPU. Its basic principle is to provide a virtual renderer (called virgl) in the host as a bridge for communication between the virtual GPU and the hardware GPU, for forwarding the rendering instructions provided by the virtual GPU to the hardware GPU and driving the hardware GPU to perform rendering. Under this driving technology framework, the source host can manage and schedule the hardware rendering process through the virtual renderer virgl. Moreover, the virglrenderer driving technology framework also provides support for OpenGL. OpenGL (Open Graphics Library) is a cross-language and cross-platform application programming interface for rendering vector graphics. Based on this, the virtual renderer virgl can provide a set of general OpenGL interfaces to the virtual GPU, and the virtual GPU can call these OpenGL interfaces to drive the hardware GPU to perform rendering. Moreover, based on the cross-language and cross-platform capabilities of the OpenGL interface, in this embodiment, various models of hardware GPUs can be supported for rendering acceleration by the target virtual machine, without being limited by the private driver of the hardware GPU. As mentioned earlier, the technical concept provided in this embodiment takes the hardware rendering process as the main body. Therefore, when using various models of hardware GPUs for rendering acceleration, the technical concept provided in this embodiment can be used to complete the state migration of the hardware rendering process. In summary, in this embodiment, a virtual machine migration method is proposed for a virtual machine using a hardware GPU for rendering acceleration. After receiving a hot migration trigger instruction, the source host can control the target virtual machine to stop sending instructions to the virtual GPU used by the target virtual machine, and wait for the execution of the rendering instructions that are still being executed to be completed. Then, the device state data corresponding to the virtual GPU used by the target virtual machine and the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU can be migrated to the destination host, to support state restoration for the virtual GPU and the hardware rendering process involved in the target virtual machine in the destination host.According to the foregoing, in the embodiment of the present disclosure, instead of replaying the instructions on the destination host, the detailed migration of the data in the above aspects is inserted into the original live migration process of the virtual machine, so that the destination host can ensure that each subject related to rendering acceleration of the target virtual machine can be restored to the state before migration based on the data in the above aspects, which can effectively improve the migration efficiency of such virtual machines. FIG. 3 is a flowchart of another virtual machine migration method provided by an exemplary embodiment of the present disclosure. Referring to FIG. 3, the method is still executed by the source host. Referring to FIG. 3, the virtual machine migration method executed by the source host can include: step 300, in response to a live migration trigger instruction for the target virtual machine, controlling the target virtual machine to stop sending rendering instructions to the virtual GPU used by the target virtual machine; step 301, during the live migration of the target virtual machine, if it is monitored that the rendering instructions received by the virtual GPU have been executed, migrating the device state data corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the virtual GPU; step 302, if the virtual GPU is generated by virtualizing the hardware GPU using the VirtIO technology, migrating the driver state data generated by the device driver of the virtual GPU in the target virtual machine to the destination host to support the destination host to restore the state of the device driver; and step 303, migrating the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the hardware rendering process. The step 300 can refer to the description of step 100 in the embodiment shown in FIG. 1, the step 301 can refer to the description of step 101 in the embodiment shown in FIG. 1, and the step 303 can refer to the description of step 102 in the embodiment shown in FIG. 1, which will not be repeated here. In the embodiment, compared with the virtual machine migration method provided in the embodiment shown in FIG. 1, step 302 is added. In step 302, it is proposed that the virtual GPU used by the target virtual machine can be generated by virtualizing the hardware GPU using the VirtIO technology. As can be known from the foregoing description of the VirtIO technology, the VirtIO technology adopts a front-end and back-end architecture, in the embodiment, the virtual GPU used by the target virtual machine is the back-end device in the front-end and back-end architecture, and the VirtIO technology also sets a device driver for the generated virtual GPU and deploys the device driver in the target virtual machine. FIG. 4 is a schematic diagram of a rendering acceleration principle based on the VirtIO technology and the Virglrenderer technology.Referring to FIG. 4, the device driver provided by the VirtIO technology for the virtual GPU is deployed in the target virtual machine, and the application in the target virtual machine can send a rendering instruction to the device driver, the device driver can transmit the rendering instruction to the virtual GPU, and the virtual GPU can transmit the rendering instruction to the virtual renderer virgl provided by the Virgl renderer technology, and the virtual renderer virgl drives the hardware GPU to perform rendering according to the rendering instruction. Based on this, in the step 302, the source host can migrate the driving state data generated by the device driver of the virtual GPU in the target virtual machine to the destination host after monitoring that the rendering instruction received by the virtual GPU is executed, so as to support the target host to restore the state of the device driver. In actual application, as mentioned in the foregoing embodiment, the driving state data to be migrated in the step 302 is part of the memory data corresponding to the target virtual machine, and therefore, the migration of the driving state data in the step 302 can be implemented based on the migration logic of the memory data. In an exemplary implementation manner: the driving state data can be migrated to the destination host as the memory dirty page data corresponding to the target virtual machine after monitoring that the rendering instruction received by the virtual GPU is executed. The memory dirty page data refers to the changed memory data, and by marking the driving state data as the memory dirty page data after monitoring that the rendering instruction received by the virtual GPU is executed, the migration can be implemented after the driving state data is stable and unchanged, which can save the data migration work during the change of the driving state data, thereby reducing the migration workload. Of course, this is not limited in the embodiment, and according to the existing memory data migration logic, it can also be ensured that the driving state data migrated to the destination host is the driving state data after the rendering instruction received by the virtual GPU is executed. In summary, in the embodiment, it is proposed that when the VirtIO technology is used to virtualize the hardware GPU, the driving state data generated by the device driver of the virtual GPU in the target virtual machine is taken as the data to be migrated, so as to support the destination host to restore the state of the device driver of the virtual GPU. The driving state data is part of the memory data corresponding to the target virtual machine, and can be fully migrated to the destination host. FIG. 5 is a flowchart of another virtual machine migration method provided by an exemplary embodiment of the present disclosure.Referring to FIG. 5, the method is also applicable to the source host, and the method can include: step 500, in response to a hot migration trigger instruction for the target virtual machine, controlling the target virtual machine to stop sending rendering instructions to a virtual GPU used by the target virtual machine; step 501, during the hot migration of the target virtual machine, if it is monitored that the rendering instruction received by the virtual GPU has been executed, migrating the device state data corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the virtual GPU; step 502, if the virtual GPU is generated by virtualizing a hardware GPU using VirtIO technology, migrating the driver state data generated by the device driver of the virtual GPU in the target virtual machine to the destination host to support the destination host to restore the state of the device driver; step 503, judging whether the target virtual machine and the virtual GPU use shared memory in the source host, if yes, executing step 504, if not, step 504 need not be executed; step 504, reading the address mapping relationship between the target virtual machine and the virtual GPU from the virtual machine manager in the source host; generating address binding information for the rendering resources used in the hardware rendering process, the address binding information being used to describe the guest physical memory address occupied by the rendering resources in the shared memory; carrying the address mapping relationship and the address binding information to the destination host to support the destination host to update the address mapping relationship and the rendering resources; step 505, migrating the rendering data and the state data generated by the hardware rendering process corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the hardware rendering process. It is worth mentioning that, in order to facilitate illustration, the specific content in steps 500-505 in FIG. 5 is abbreviated, but this should not cause ambiguity or loss of protection scope. Compared with the virtual machine migration method provided in FIG. 3, in this embodiment, steps 503 and 504 are added, in this embodiment, the shared memory mechanism is supported between the virtual GPU and the target virtual machine to accelerate data transmission, so as to improve the rendering acceleration performance. For example, in the foregoing VirtIO technology, the VirtIO-GPU and the virtual machine usually use the shared memory mechanism. Referring to FIGS. 2 and 4, the target virtual machine is located in the guest operating system (Guest) in the source host, and the virtual machine manager VMM is located in the host operating system (Host), under the shared memory mechanism, the physical memory allocated for the virtual GPU can be shared by the target virtual machine and the virtual GPU, and both parties can access the shared memory through the direct memory access DMA.Under the shared memory mechanism, the virtual machine manager VMM maintains an address mapping relationship between the target virtual machine and the virtual GPU for the shared memory. In actual application, the address mapping relationship records a mapping relationship between a guest physical memory address (GPA) used by the target virtual machine and a host virtual memory address (HVA) used by the virtual GPU. Based on the shared memory mechanism, the virtual machine can access the memory space of the virtual GPU through the DMA mode without the participation of the CPU of the source host. For this purpose, the embodiment proposes that if the target virtual machine and the virtual GPU use the shared memory in the source host, the source host can also migrate the above address mapping relationship maintained for the shared memory to the destination host. In addition, in addition to proposing to migrate the address mapping relationship, the embodiment also proposes that the source host generates address binding information for rendering resources used in the hardware rendering process. The address binding information records a guest physical memory address GPA occupied by the rendering resource in the shared memory. The rendering resource can be understood as a resource object that needs to be used in the execution of the rendering process (i.e., the process of running the hardware rendering process). Common rendering resources can include but are not limited to textures, buffers, shaders, etc., and no more examples are given here. The rendering resource usually defines a plurality of attributes, and under the shared memory mechanism, the rendering resource defines an attribute for recording a DMA address, where the DMA address specifically refers to a host virtual memory address HVA in the shared memory, rather than a guest physical memory address GPA. This is mainly because the rendering resource is created in the hardware rendering process, and the management and scheduling of the hardware rendering process are also located in the Host system, so the address recorded in the rendering resource is HVA, and not GPA. The inventors found in the research process that after the target virtual machine is migrated to the destination host, the destination host will use the guest physical memory address GPA in the target virtual machine for the shared memory, but will reallocate a host virtual memory address HVA for the shared memory. This will cause the HVA recorded in the rendering resource and the HVA reallocated by the destination host for the same memory location in the shared memory to be inconsistent, which may further cause the rendering resource to be unable to be normally used. For this purpose, the embodiment proposes to add processing logic for generating address binding information for the rendering resource in the source host.An example generation logic can be: querying a target host virtual memory address HVA corresponding to rendering data used by a target rendering resource in shared memory; determining a target guest physical memory address GPA mapped by the target host virtual memory address HVA based on an address mapping relationship, and recording the guest physical memory address GPA occupied by the target rendering resource in the shared memory; based on this, the address binding information of the target rendering resource can be generated in the process of migrating the target virtual machine. In the example generation logic, the source host can record the GPA occupied by the rendering resource in the shared memory (determined by querying the address mapping table), and when virtual machine migration is needed, the source host can generate the address binding information of the rendering resource based on the information recorded by the rendering resource. It should be understood that in this embodiment, the source host does not intervene in the address recorded in the rendering resource, but generates address binding information for the rendering resource outside the rendering resource. Accordingly, in this embodiment, it is proposed that in the case of using shared memory between the target virtual machine and the virtual GPU, the source host will also migrate the address mapping relationship maintained for the shared memory in the process of hot migrating the target virtual machine; the source host will also generate address binding information for the rendering resource used by the hardware rendering process to be migrated, and migrate it. Based on the two aspects of data migration proposed in this embodiment, the destination host can update the migrated address mapping information according to the HVA newly allocated for the shared memory after restoring the shared memory, and update the HVA recorded in the migrated rendering resource, so as to ensure that the migrated shared memory and rendering resource can be normally used. In the above or the following embodiments, a plurality of implementation manners can be used to migrate the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU to the destination host. An optional implementation manner is provided below. First, the processing logic of the state data in this implementation manner is described. In this optional implementation manner, the state data is proposed: if the virtual GPU calls the hardware GPU through the virtual rendering component provided by the source host, the source host can migrate the context information maintained in the virtual rendering component for the hardware rendering process corresponding to the virtual GPU to the destination host. Here, the virtual rendering component is a functional component in the source host for managing and scheduling the hardware rendering process. In different driver technology frameworks, the implementation form of the virtual rendering component may be different.As mentioned above, the Virglrenderer driver architecture is preferably used to support the virtual GPU to call the hardware GPU, in which the virtual renderer (virgl) is a virtual rendering component provided by the source host. The virtual rendering component maintains context information for each hardware rendering process, for example, the virgl context is maintained for each hardware rendering process in the virtual renderer (virgl), which is essentially an OpenGL context, and the virtual renderer (virgl) encapsulates the OpenGL context to generate the virgl context. The virtual renderer (virgl) is essentially a state machine, and a typical interpretation of the virgl context states that the virgl context records all information and states required for rendering. In other words, the virgl context records all information and states required for the corresponding hardware rendering process. Based on this, in this optional implementation, it is proposed that the context information maintained by the virtual rendering component for the hardware rendering process is used as the state data that needs to be migrated. Obviously, the content recorded in the context information is sufficient to support the destination host to restore the state of the hardware rendering process. In addition, in this optional implementation, the state data is also proposed that the source host can also migrate the global state information maintained by the virtual rendering component for the virtual GPU to the destination host. The global state information records the hardware rendering process being executed by the virtual GPU when the migration trigger instruction is received. In this optional implementation, the global state information maintained for the virtual GPU is also used as the state data that needs to be migrated. In this way, by migrating the global state information, the destination host can restore the hardware rendering process being executed in the virtual rendering component to the hardware rendering process recorded in the global state information, which can restore the work progress in the virtual rendering component to the state before migration. Based on this, taking the aforementioned Virglrenderer driver architecture as an example, in this optional implementation, the source host can migrate the following two aspects of data as the state data that needs to be migrated for the hardware rendering process:

[0004] 1) virgl context;

[0005] 2) Global state information. Here, a brief description of the virgl context is given. In the embodiment, one or more hardware rendering processes can be created for the virtual GPU, and the virtual rendering component maintains context information for each hardware rendering process. Each context information can include two types of information: one is main context information, and the other is sub-context information. The main context information usually records some attributes and states shared by the process, and multiple sub-context information can share the main context information. A hardware rendering process can include multiple rendering tasks, and each rendering task can be regarded as a rendering thread in the hardware rendering process. The virtual rendering component maintains the sub-context information for each rendering thread. In this way, multiple sub-context information share the main context information, and it can also be interpreted that multiple rendering threads can share the rendering resources in the rendering process. Based on this, in the implementation mode, for a virgl context, at least the following several migration aspects can be included:

[0006] 1) Identification of the virgl context;

[0007] 2) Main context in the virgl context;

[0008] 3) each sub-context in the virgl context. By migrating both the virgl context and the global state information to the destination host, the destination host can restore the hardware rendering process based on the virgl context and the global state information, and the destination host can restore the work progress of the virtual renderer (virgl) to the hardware rendering process recorded in the global state information. The following describes the processing logic for rendering data in this implementation. In this implementation, the following is proposed for rendering data: determining the target rendering resource used in the hardware rendering process; collecting rendering data of the target rendering resource saved in the hardware GPU and / or saved in the physical memory of the source host; and migrating the collected rendering data to the destination host. In this implementation, each context information collected above can be traversed to parse the rendering resource used in the hardware rendering process from the context information. For a rendering resource, the rendering data generated in the process of being used by the hardware rendering process can be saved in the hardware GPU or saved in the physical memory of the source host (for example, the shared memory described above). For the rendering data saved in the physical memory of the source host, the source host can perform a read operation on the physical memory to migrate the rendering data to the destination host. For the rendering data saved in the hardware GPU, the source host is unaware of the storage location of the rendering data in the hardware GPU due to the isolation of the memory address spaces, and thus the source host cannot directly collect the rendering data. Therefore, in this implementation, the following is further proposed for the rendering data saved in the hardware GPU: if a first rendering resource in the target rendering resource has rendering data saved in the hardware GPU, the source host can select an access interface required to be called for the first rendering resource based on a mapping relationship between the type of the rendering resource and the access interface, the access interface defining access logic for accessing preset data from the hardware GPU; and the rendering data saved in the hardware GPU for the first rendering resource is read by using the selected access interface. The access interface is defined in this embodiment, and a communication protocol can be predefined between the access interface and the hardware GPU to support that an access instruction initiated by the access interface can be correctly responded by the hardware GPU. For different types of rendering resources, the types of rendering data required to be accessed can be different, and the access logic required to be used can also be different. Therefore, this embodiment proposes that the access interface is designed for different types of rendering resources. Specifically, one or more access interfaces can be designed for one type of rendering resource to support that each type of rendering data required to be accessed from the hardware GPU for the type of rendering resource.It can be understood that, after the type of the rendering resource is known and the type of the rendering data required to be accessed is known, the usage of components such as a virtual renderer (virgl) can be queried to find the memory access logic of the hardware GPU in the running process of the hardware rendering process, the access path can be found from the memory access logic, and the access logic in the access interface can be designed based on the access path. The design of the access logic in the access interface belongs to the engineering implementation problem, and can be flexibly designed in actual application, which is not expanded and described here. Of course, the above optional implementation mode is exemplified, in this embodiment, the source host can also use other implementation modes to migrate the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU to the destination host, for example, a part of the data can be filtered from the context information of the hardware rendering process for migration, without the need to migrate all the context information. No more examples are given here. In addition, in some driving technology framework, the virtual GPU can also independently manage and schedule the hardware rendering process without relying on the foregoing virtual rendering component, in this case, the virtual GPU needs to have the driving capability for different types of hardware GPU, as mentioned in the foregoing, which is only different from the driving technology framework of driving the hardware GPU through the virtual rendering component in terms of the way of managing and scheduling the hardware rendering process. In this case, the state data corresponding to the hardware rendering process is still maintained by the source host (specifically, the virtual machine manager VMM in the source host), therefore, the source host can still obtain the state data corresponding to the hardware rendering process without obstacles, and the rendering data is also the same, which is not expanded and described here. In summary, in this embodiment, the source host can collect the rendering data and state data of the hardware rendering process to be migrated in detail and migrate them to the destination host, which can provide sufficient data basis for the destination host, thereby supporting the destination host to restore the state of the hardware rendering process, restore the state of the virtual rendering component, and the like. The above is a description of the virtual machine migration method on the source host side. It should be understood that, with reference to FIG. 5, after the data migration of the foregoing aspects is completed on the source host side, and the source host side has also reached the shutdown time according to the existing migration logic (for example, when the memory dirty page data amount of the latest round of memory data migration is lower than the preset threshold value by using the pre-copy mode), the target virtual machine on the source host can be shut down. The virtual machine migration method on the destination host side is adapted to the virtual machine migration method on the source host side, which can be understood as the reverse process of the virtual machine migration method on the source host side. The virtual machine recovery process on the destination host side is described below. FIG. 6 is a flowchart of a virtual machine migration method provided by another exemplary embodiment of the present disclosure.The virtual machine migration method is applicable to a destination host to which the target virtual machine needs to be migrated. Similarly, the destination host can also integrate a virtual machine migration device, and the virtual machine migration method is executed by the virtual machine migration device, which can be implemented as software, hardware, or a combination of software and hardware. Referring to FIG. 6, the virtual machine migration method can include: step 600, in response to a recovery instruction for the target virtual machine, initializing the target virtual machine; step 601, after the initialization is completed, restoring the state of a virtual GPU required to be used by the target virtual machine according to device state data corresponding to the virtual GPU migrated from a source host of the target virtual machine; and step 602, restoring the state of a hardware rendering process according to rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU migrated from the source host. In this embodiment, the destination host also uses the existing migration logic in the virtual machine live migration technology, for example, initializing the virtual machine, allocating physical memory to the virtual machine, and the like, and the conventional work aspects are not described in detail. Referring to FIG. 6, it is proposed in step 601 that after the initialization of the target virtual machine is completed, the state of each subject related to rendering acceleration can be restored. In response to the device state data corresponding to the virtual GPU proposed in FIG. 1 on the source host side, it is proposed in step 601 in this embodiment that the destination host can restore the state of the virtual GPU required to be used by the target virtual machine according to the device state data corresponding to the virtual GPU migrated from the source host of the target virtual machine. In response to the rendering data and state data corresponding to the hardware rendering process proposed in FIG. 1 on the source host side, it is proposed in step 602 that the destination host can restore the state of the hardware rendering process according to the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU migrated from the source host. It should be understood that the state restoration in this embodiment can be understood as storing the data migrated from the source host in the destination host, so as to ensure that each subject related to rendering acceleration can use the data as before the migration in the destination host, so that the state of each subject is as before the migration through the data. It can be known that in this embodiment, instruction playback is no longer required in the destination host, but the state restoration problem is converted into a data migration problem, and the device state data of the virtual GPU and the rendering data and state data corresponding to the hardware rendering process migrated from the source host are stored in the destination host according to the storage mode in the source host.With the device state data as an example, since the device state data migrated from the source host describes the working state of the virtual GPU at the migration moment, the target host can save the migrated device state data to the storage location recognized by the virtual GPU, so as to restore the state of the virtual GPU to the working state at the migration moment. The state restoration principle of the hardware rendering process is also the same. In summary, in this embodiment, for a virtual machine using a hardware GPU for rendering acceleration, after receiving a hot migration trigger instruction, the source host can control the target virtual machine to stop sending instructions to the virtual GPU used by the target virtual machine, and after waiting for the execution of the rendering instructions in progress to be completed, the source host can start migrating the device state data corresponding to the virtual GPU used by the target virtual machine and the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU to the target host, to support state restoration of the virtual GPU and the hardware rendering process involved in the target virtual machine in the target host. Accordingly, in the embodiment of the present disclosure, it is no longer necessary to perform instruction playback in the target host, but to complete the detailed migration of the above-mentioned data during the original hot migration process of the virtual machine, so that the target virtual machine can guarantee that each subject related to rendering acceleration can be restored to the state before migration based on the above-mentioned data, which can effectively improve the migration efficiency of such virtual machines. It can be known that in the embodiment, the source host and the target host of the target virtual machine can cooperate with each other, and through detailed data migration operations between the two, the related subjects involved in the rendering acceleration process using the hardware GPU by the target virtual machine can be guaranteed to be restored to the state before migration. In an optional embodiment, the same GPU virtualization technology is used in the target host and the source host. If the virtual GPU is generated by using the VirtIO technology to virtualize the hardware GPU, the target host can restore the state of the device driver corresponding to the virtual GPU used by the target virtual machine according to the driver state data migrated from the source host. Since the device driver corresponding to the virtual GPU is still deployed in the target virtual machine in the target host, the driver state data corresponding to the device driver is stored as part of the memory data in the target host according to the existing memory data storage scheme, so as to ensure that the device driver can access the required driver state data without obstacles, thereby ensuring that the device driver can be restored to the state before migration. FIG. 7 is a flowchart of another virtual machine migration method provided by another exemplary embodiment of the present disclosure.Referring to FIG. 7, the method is applicable to a destination host, and the method can include: step 700, in response to a recovery instruction for a target virtual machine, initializing the target virtual machine; step 701, after the initialization is completed, restoring a state of a virtual GPU required to be used by the target virtual machine according to device state data corresponding to the virtual GPU migrated from a source host of the target virtual machine; step 702, if the virtual GPU is generated by virtualizing a hardware GPU using VirtIO technology, restoring a state of a device driver corresponding to the virtual GPU used by the target virtual machine according to driver state data migrated from the source host; step 703, determining whether the target virtual machine and the virtual GPU require shared memory, if yes, performing step 703, otherwise, not performing step 703; step 704, allocating physical memory as shared memory from the destination host; in a host virtual memory address space of the destination host, re-allocating a host virtual memory address HVA for the shared memory, a guest physical memory address GPA used by the target virtual machine on the shared memory remains unchanged; updating address mapping relationship for the shared memory migrated from the source host according to the re-allocated host virtual memory address HVA; step 705, restoring a state of a hardware rendering process according to rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU migrated from the source host. For the convenience of illustration, the specific content in steps 700-704 in FIG. 7 is abbreviated, but this should not cause ambiguity or loss of protection scope. As described above, the address mapping relationship mentioned in step 703 is used to record the mapping relationship between the guest physical memory address GPA used by the target virtual machine and the host virtual memory address HVA used by the virtual GPU for the shared memory. In the destination host, the shared memory is re-allocated with the HVA, therefore, the destination host can replace the HVA in the address mapping relationship migrated from the source host. To ensure that the target virtual machine and the virtual GPU after migration can normally use the shared memory, and will not cause the problem of being unable to use due to address mapping relationship error.In addition, referring to FIG. 7, in step 704, the destination host also parses the target guest physical memory address occupied by the target rendering resource in the shared memory from the address binding information corresponding to the rendering resource migrated from the source host in correspondence with the rendering resource migrated from the source host; determines the target host virtual memory address mapped by the target guest physical memory address based on the updated address mapping relationship; and updates the host virtual memory address recorded in the target rendering resource to the target host virtual memory address. The target rendering resource is any rendering resource used by the hardware rendering process. To this end, the destination host can traverse all rendering resources used by the migrated hardware rendering process, and re-determine the host virtual memory address HVA occupied in the shared memory for each rendering resource, thereby updating the attribute for recording the DMA address in each rendering resource. This can ensure that the migrated hardware rendering process can normally access the required rendering data when using the rendering resource, thereby avoiding the problem of being unable to normally use the rendering resource due to the address. Optionally, if the virtual GPU on the source host side invokes the hardware GPU through the virtual rendering component, the destination host also provides a virtual rendering component for the virtual GPU to invoke. On this basis, if the context information corresponding to the hardware rendering process of the virtual GPU is contained in the state data migrated from the source host, the destination host can restore the context information for the hardware rendering process of the virtual GPU in the virtual rendering component provided by the destination host. The context information is usually stored in the physical memory allocated for the virtual rendering component in the source host, and here, the context information can be stored in the physical memory allocated for the virtual rendering component in the destination host, so as to ensure that the virtual rendering component provided by the destination host can normally use the context information. Since the context information describes the working state of the hardware rendering process at the migration time, storing the context information in the destination host can restore the hardware rendering process to the state before migration. Optionally, if the global state information is also contained in the state data, the destination host can restore the virtual rendering component to the target hardware rendering process recorded in the global state information, and the target hardware rendering process is the hardware rendering process corresponding to the virtual GPU being executed by the source host when the migration trigger instruction is received. In this way, in the destination host, the working progress of the virtual rendering component can be restored to the state before migration, and after the target virtual machine is re-run, the target hardware rendering process will continue to run in the destination host.In the above or below embodiments, the state recovery process of the hardware rendering process can include: traversing the collected context information to determine the target rendering resources used by the hardware rendering process in the virtual rendering component; saving the rendering data used by the target rendering resources to the hardware GPU and / or the physical memory of the destination host. That is, for the rendering data corresponding to the hardware rendering process, part of the rendering data is saved to the physical memory of the destination host (for example, the above-mentioned reallocated shared memory) according to the storage mode in the source host; another part of the rendering data can be saved to the hardware GPU. Optionally, for the target rendering data that needs to be saved to the hardware GPU, if the second rendering resource in the target rendering resource needs to save the rendering data to the hardware graphics processor, the destination host can select the required access interface for the second rendering resource based on the mapping relationship between the type of the rendering resource and the access interface, and the access interface defines the access logic for accessing the preset data from the hardware GPU; the rendering data required to be saved in the hardware GPU by the second rendering resource is written to the hardware GPU by using the selected access interface. It should be noted that the technical details in the above-mentioned virtual machine migration method corresponding to the destination host can refer to the reverse process of the above-mentioned virtual machine migration method corresponding to the source host, which will not be expanded and described here. However, this should not cause any loss to the protection scope of the present disclosure. It should be noted that in some of the processes described in the above embodiments and accompanying drawings, a plurality of operations appear in a specific order, but it should be clearly understood that these operations can be executed or executed in parallel without the order in which they appear in this text. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and these operations can be executed in sequence or in parallel. FIG. 8 is a structural schematic diagram of a computing device provided by another exemplary embodiment of the present disclosure. As shown in FIG. 8, the computing device can include a memory 80, a processor 81, and a communication component 82. The computing device can serve as a source host in a virtual machine migration process.In this case, the processor 81, coupled with the memory and the communication component 82, is configured to execute a computer program in the memory 80 for: in response to a hot migration trigger instruction for the target virtual machine, controlling the target virtual machine to stop sending rendering instructions to a virtual GPU used by the target virtual machine; during the hot migration of the target virtual machine, if it is monitored that the execution of the rendering instructions received by the virtual GPU is completed, migrating the device state data corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the virtual GPU; and migrating the rendering data and state data generated by a hardware rendering process corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the hardware rendering process. In an optional embodiment, the processor 81 can also be used for: if the virtual GPU is generated by virtualizing a hardware GPU using VirtIO technology, after monitoring that the execution of the rendering instructions received by the virtual GPU is completed, migrating the driver state data generated by the device driver of the virtual GPU in the target virtual machine to the destination host to support the destination host to restore the state of the device driver. In an optional embodiment, when the processor 81 migrates the driver state data generated by the device driver of the virtual GPU in the target virtual machine to the destination host, it can be specifically used for: migrating the driver state data as memory dirty page data corresponding to the target virtual machine to the destination host. In an optional embodiment, the processor 81 can also be used for: if the target virtual machine and the virtual GPU use shared memory in the source host, reading the address mapping relationship between the target virtual machine and the virtual GPU from the virtual machine manager in the source host; generating address binding information for the rendering resources used in the hardware rendering process, the address binding information being used to describe the client physical memory addresses occupied by the rendering resources in the shared memory; and migrating the address mapping relationship and the address binding information to the destination host to support the destination host to update the address mapping relationship and the rendering resources.In an optional embodiment, the address mapping relationship is used to record a mapping relationship between a guest physical memory address used by the target virtual machine and a host virtual memory address used by the virtual GPU; the processor 81 can also be configured to: query a target host virtual memory address occupied by rendering data used by a target rendering resource in the shared memory; determine a target guest physical memory address mapped to by the target host virtual memory address based on the address mapping relationship, and record the target guest physical memory address as a guest physical memory address occupied by the target rendering resource in the shared memory; wherein the target rendering resource is any rendering resource used in the hardware rendering process. In an optional embodiment, when the processor 81 migrates state data generated by the hardware rendering process corresponding to the virtual GPU to the destination host, the processor 81 can be specifically configured to: if the virtual GPU calls a hardware GPU through the virtual rendering component, migrate context information maintained in the virtual rendering component for the hardware rendering process corresponding to the virtual GPU to the destination host; and / or migrate global state information maintained in the virtual rendering component for the virtual GPU to the destination host, the global state information recording an executing hardware rendering process corresponding to the virtual GPU when the migration trigger instruction is received. In an optional embodiment, when the processor 81 migrates rendering data generated by the hardware rendering process corresponding to the virtual GPU to the destination host, the processor 81 can be specifically configured to: determine a target rendering resource used in the hardware rendering process; collect rendering data of the target rendering resource saved in the hardware GPU and / or rendering data maintained in the physical memory of the source host; and migrate the collected rendering data to the destination host. In an optional embodiment, when the processor 81 collects rendering data of the target rendering resource saved in the hardware GPU, the processor 81 can be specifically configured to: if a first rendering resource in the target rendering resource has rendering data saved in the hardware GPU, select an access interface required to be called for the first rendering resource based on a mapping relationship between a type of the rendering resource and the access interface, the access interface defining access logic for accessing preset data from the hardware GPU; and read the rendering data of the first rendering resource saved in the hardware GPU by using the selected access interface.In an optional embodiment, the hardware GPU on the source host is virtualized by using virtualization technology to generate the virtual GPU; the rendering instruction uses openGL instruction; the virtual GPU invokes the corresponding hardware GPU to perform the hardware rendering process by using virglrenderer technology, and the virglrenderer component in the virglrenderer technology is used as the virtual rendering component to forward the rendering instruction received by the virtual GPU to the hardware GPU to drive the hardware GPU to perform rendering according to the openGL instruction. It is worth mentioning that the technical details in the above embodiments of the computing device can refer to the related description in the foregoing method embodiments of the source host side, and will not be described here again to save space, but this should not cause the loss of the protection scope of the present disclosure. The computing device can be used as the source host in the virtual machine migration process. In this case, the processor 81 coupled with the memory and the communication component 82 is configured to execute the computer program in the memory 80 to: in response to a recovery instruction for the target virtual machine, initialize the target virtual machine; after the initialization is completed, restore the state of the virtual GPU required to be used by the target virtual machine according to the device state data corresponding to the virtual GPU migrated from the source host of the target virtual machine; and restore the state of the hardware rendering process according to the rendering data and the state data generated by the hardware rendering process corresponding to the virtual GPU migrated from the source host. In an optional embodiment, the processor 81 can also be used to: if the virtual GPU is generated by virtualizing the hardware GPU by using VirtIO technology, restore the state of the device driver corresponding to the virtual GPU in the target virtual machine according to the driving state data migrated from the source host. In an optional embodiment, the processor 81 can also be used to: if the target virtual machine and the virtual GPU need to use shared memory, allocate physical memory as the shared memory from the destination host; re-allocate host virtual memory addresses for the shared memory in the host virtual memory address space of the destination host, and the guest physical memory address used by the target virtual machine on the shared memory remains unchanged; and update the address mapping relationship for the shared memory migrated from the source host according to the re-allocated host virtual memory addresses.In an optional embodiment, the processor 81 can also be configured to: for a target rendering resource used by the hardware rendering process, parse a target guest physical memory address occupied by the target rendering resource in the shared memory from address binding information corresponding to the target rendering resource migrated from the source host; determine a target host virtual memory address mapped by the target guest physical memory address based on the updated address mapping relationship; update a host virtual memory address recorded in the target rendering resource to the target host virtual memory address; wherein the target rendering resource is any rendering resource used by the hardware rendering process. In an optional embodiment, when restoring the state of the hardware rendering process according to the state data generated by the hardware rendering process corresponding to the virtual GPU migrated from the source host, the processor 81 can be specifically configured to: if the virtual GPU calls the hardware GPU through the virtual rendering component provided by the destination host, then according to context information corresponding to the hardware rendering process included in the state data, restore the context information for the hardware rendering process corresponding to the virtual GPU in the virtual rendering component provided by the destination host; and / or according to global state information included in the state data, restore the virtual rendering component to a target hardware rendering process recorded in the global state information, the target hardware rendering process being the hardware rendering process being executed by the virtual GPU when the source host receives the migration trigger instruction. In an optional embodiment, when restoring the state of the hardware rendering process according to the rendering data generated by the hardware rendering process corresponding to the virtual GPU migrated from the source host, the processor 81 can be specifically configured to: traverse each context information collected to determine a target rendering resource used by the hardware rendering process in the virtual rendering component; save rendering data used by the target rendering resource to the hardware GPU and / or the physical memory of the destination host.In an optional embodiment, when the processor 81 saves the rendering data used by the target rendering resource into the hardware GPU, the processor 81 can specifically be configured to: if a second rendering resource in the target rendering resource needs to save rendering data into the hardware GPU, select an access interface required to be called for the second rendering resource based on a mapping relationship between the type of the rendering resource and the access interface, the access interface defining access logic for accessing preset data from the hardware GPU; and write the rendering data required to be saved in the hardware GPU by the second rendering resource into the hardware GPU by using the selected access interface. It should be noted that the technical details in the above embodiments of the computing device can refer to the related descriptions in the foregoing method embodiments on the host side for the purpose of saving space, and will not be described herein again, but this should not cause a loss of the protection scope of the present disclosure. Further, as shown in FIG. 8, the computing device can further include other components such as a power supply component 83. The partial components shown in FIG. 8 do not mean that the computing device only includes the components shown in FIG. 8. The embodiments of the present disclosure further provide a virtual machine migration system including a source host where a target virtual machine is located and a destination host to which the target virtual machine needs to be migrated. The source host can be configured to: in response to a hot migration trigger instruction for the target virtual machine, control the target virtual machine to stop sending rendering instructions to a virtual GPU used by the target virtual machine; during the hot migration of the target virtual machine, after monitoring that the rendering instructions received by the virtual GPU are executed, migrate device state data corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the virtual GPU; and migrate rendering data and state data generated by a hardware rendering process corresponding to the virtual GPU to the destination host to support the destination host to restore the state of the hardware rendering process. The destination host can be configured to: in response to a recovery instruction for the target virtual machine, initialize the target virtual machine; after the initialization is completed, restore the state of a virtual GPU required to be used by the target virtual machine according to the device state data corresponding to the virtual GPU migrated from the source host of the target virtual machine; and restore the state of the hardware rendering process according to the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU migrated from the source host. The technical details in the source host and the destination host can refer to the foregoing method embodiments on the host side and the method embodiments on the host side for the purpose of saving space, and will not be described herein again, but this should not cause a loss of the protection scope of the present disclosure.Accordingly, in the virtual machine migration system, for a virtual machine using a hardware GPU for rendering acceleration, a virtual machine migration method is provided. After receiving a hot migration trigger instruction, the source host can control the target virtual machine to stop sending instructions to the virtual GPU used by the target virtual machine, and after the execution of the rendering instructions in progress is completed, the device state data corresponding to the virtual GPU used by the target virtual machine and the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU can be migrated to the destination host to support state recovery for the virtual GPU and the hardware rendering process involved in the target virtual machine in the destination host. Accordingly, in the embodiments of the present disclosure, instruction playback is no longer required on the destination host, and the detailed migration of the above-mentioned data is completed during the original hot migration process of the virtual machine, so that the subjects related to rendering acceleration (such as the virtual GPU and the hardware rendering process) of the target virtual machine can be restored to the state before migration on the destination host side, which can effectively improve the migration efficiency of such virtual machines. Accordingly, the embodiments of the present disclosure also provide a computer-readable storage medium storing a computer program, which can implement the steps in the above method embodiments when executed. Accordingly, the embodiments of the present disclosure also provide a computer program product, which contains a computer program that can implement the steps in the above method embodiments when executed. The memory in the above FIG. 8 is configured to store a computer program, and can be configured to store other various data to support operations on the computing platform. Examples of these data include instructions for any application or method operating on the computing platform, contact data, phonebook data, messages, images, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The communication component in the above FIG. 8 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G, etc. mobile communication network, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel.In one example embodiment, the communication component processor 81 can also be used for a near field communication (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies. The power component in FIG. 8 provides power to various components of the device in which the power component is located. The power component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device in which the power component is located. As will be appreciated by one skilled in the art, embodiments of the present disclosure can be provided as a method, system, or computer program product. Accordingly, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, and the like) embodying computer program instructions. The present disclosure is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks.It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... " does not exclude the existence of additional elements of the same kind in the process, method, article, or apparatus that includes the stated elements. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal. The above only describes the embodiments of the present disclosure and does not limit the present disclosure. The present disclosure can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure. Industrial applicability The present disclosure proposes a virtual machine migration method for virtual machines using hardware GPU for rendering acceleration. After receiving the hot migration trigger instruction, the source host can control the target virtual machine to stop sending instructions to the virtual GPU used by the target virtual machine, and after the execution of the rendering instruction in progress is completed, the device state data corresponding to the virtual GPU used by the target virtual machine and the rendering data and state data generated by the hardware rendering process corresponding to the virtual GPU can be migrated to the destination host to support state restoration for the virtual GPU and the hardware rendering process involved in the target virtual machine in the destination host. Accordingly, in the embodiment of the present disclosure, instruction playback is no longer required on the destination host, and the detailed migration of the above several aspects is completed during the original hot migration process of the virtual machine, so that the target virtual machine can be restored to the state before migration on the destination host. The main bodies (such as virtual GPU and hardware rendering process) related to rendering acceleration can be effectively improved.

Claims

22 CLAIM 1. A method for virtual machine migration, applicable to a source host where a target virtual machine is located, the method comprising: In response to a hot migration trigger instruction for the target virtual machine, the target virtual machine is controlled to stop sending rendering instructions to a virtual graphics processor used by the target virtual machine; during the hot migration of the target virtual machine, if it is monitored that the virtual graphics processor has completed execution of the received rendering instructions, device state data corresponding to the virtual graphics processor is migrated to a destination host to support state restoration of the virtual graphics processor by the destination host; The rendering data and state data generated by a hardware rendering process corresponding to the virtual graphics processor are migrated to the destination host to support state restoration of the hardware rendering process by the destination host.

2. The method of claim 1, wherein, Further comprising: if the virtual graphics processor is generated by virtualization of a hardware graphics processor using VirtIO technology, after it is monitored that the virtual graphics processor has completed execution of the received rendering instructions, drive state data generated by a device driver of the virtual graphics processor in the target virtual machine is migrated to the destination host to support state restoration of the device driver by the destination host.

3. The method according to claim 2, wherein, Migrating the drive state data generated by the device driver of the virtual graphics processor in the target virtual machine to the destination host includes: migrating the drive state data as dirty page data of memory corresponding to the target virtual machine to the destination host.

4. The method according to claim 1, wherein, Further comprising: If the target virtual machine and the virtual graphics processor use shared memory in the source host, address mapping relationship between the target virtual machine and the virtual graphics processor is read from a virtual machine manager in the source host; address binding information is generated for rendering resources used in the hardware rendering process, and the address binding information is used to describe client physical memory addresses occupied by the rendering resources in the shared memory; The address mapping relationship and the address binding information are migrated to the destination host to support updating of the address mapping relationship and the rendering resources by the destination host.

5. The method according to claim 4, wherein, The address mapping relationship is used to record mapping relationship between client physical memory addresses used by the target virtual machine and host virtual memory addresses used by the virtual graphics processor; The method further comprises: querying target host virtual memory addresses occupied by rendering data used by target rendering resources in the shared memory; based on the address mapping relationship, determining target client physical memory addresses mapped to by the target host virtual memory addresses, and recording as client physical memory addresses occupied by the target rendering resources in the shared memory; wherein the target rendering resources are any rendering resources used in the hardware rendering process.

6. The method of claim 1, wherein, Migrating the state data generated by the hardware rendering process corresponding to the virtual graphics processor to the destination host includes: if the virtual graphics processor invokes a hardware graphics processor through a virtual rendering component, migrating the virtual rendering component migrate, to the destination host, context information maintained in the virtual rendering component for the hardware rendering process corresponding to the virtual GPU; and / or migrate, to the destination host, global state information maintained in the virtual rendering component for the virtual GPU, the global state information recording a hardware rendering process being executed by the virtual GPU when the migration trigger instruction is received.

7. The method according to claim 6, wherein, migrating, to the destination host, rendering data generated by the hardware rendering process corresponding to the virtual GPU, including: determining a target rendering resource used in the hardware rendering process; collecting rendering data of the target rendering resource saved in the hardware GPU and / or saved in a physical memory of the source host; and migrating the collected rendering data to the destination host.

8. The method according to claim 7, wherein, collecting the rendering data of the target rendering resource saved in the hardware GPU, including: if a first rendering resource in the target rendering resource has rendering data saved in the hardware GPU, selecting an access interface required to be invoked for the first rendering resource based on a mapping relationship between a type of rendering resource and an access interface, the access interface defining access logic for accessing preset data from the hardware GPU; and reading the rendering data of the first rendering resource saved in the hardware GPU by using the selected access interface.

9. The method according to claim 1, wherein, virtualizing the hardware GPU on the source host by using a virtual 10 technology to generate the virtual GPU; the rendering instruction being an openGL instruction; and supporting, by using a virgl renderer technology, the virtual GPU to invoke the hardware GPU corresponding to the virtual GPU to execute the hardware rendering process, a virgl renderer component in the virgl renderer technology being a virtual rendering component that forwards the rendering instruction received by the virtual GPU to the hardware GPU to drive the hardware GPU to perform rendering according to the openGL instruction.

10. A method of virtual machine migration, adapted to a destination host to which a target virtual machine is migrated, the method comprising: in response to a recovery instruction for the target virtual machine, initializing the target virtual machine; after the initialization is completed, restoring a state of a virtual GPU required to be used by the target virtual machine according to device state data of the virtual GPU migrated from a source host of the target virtual machine; and restoring a state of a hardware rendering process corresponding to the virtual GPU according to rendering data and state data of the hardware rendering process migrated from the source host.

11. The method according to claim 10, wherein, the method further includes: if the virtual GPU is generated by virtualizing a hardware GPU by using a VirtIO technology, restoring a state of a device driver corresponding to the virtual GPU in the target virtual machine according to driving state data migrated from the source host.

12. The method according to claim 10, wherein, The method further comprises: if the target virtual machine and the virtual graphics processor need to use shared memory, allocating physical memory as the shared memory from the destination host; re-allocating a host virtual memory address for the shared memory in a host virtual memory address space of the destination host, a guest physical memory address used by the target virtual machine on the shared memory remaining unchanged; and updating an address mapping relationship for the shared memory migrated from the source host according to the re-allocated host virtual memory address. The method further comprises: for a target rendering resource used by the hardware rendering process, parsing a target guest physical memory address occupied by the target rendering resource in the shared memory from address binding information corresponding to the target rendering resource migrated from the source host; determining a target host virtual memory address mapped by the target guest physical memory address based on the updated address mapping relationship; and updating a host virtual memory address recorded in the target rendering resource to the target host virtual memory address; wherein the target rendering resource is any rendering resource used by the hardware rendering process.

13. The method according to claim 12, wherein, According to state data generated by a hardware rendering process corresponding to the virtual graphics processor migrated from the source host, state restoration is performed on the hardware rendering process, including: if the virtual graphics processor calls a hardware graphics processor through a virtual rendering component provided by the destination host, restoring context information of the hardware rendering process corresponding to the virtual graphics processor in the virtual rendering component according to context information included in the state data; and / or restoring the virtual rendering component to a target hardware rendering process recorded in global state information included in the state data, the target hardware rendering process being a hardware rendering process being executed by the virtual graphics processor when the source host receives a migration trigger instruction.

14. The method according to claim 10, wherein, According to rendering data generated by a hardware rendering process corresponding to the virtual graphics processor migrated from the source host, state restoration is performed on the hardware rendering process, including: traversing each context information collected to determine a target rendering resource used by the hardware rendering process in the virtual rendering component; and saving rendering data used by the target rendering resource to the hardware graphics processor and / or physical memory of the destination host.

15. The method according to claim 14, wherein, ​ 16. The method according to claim 15, wherein, The rendering data used by the target rendering resource is saved into the hardware graphics processor, including: if a second rendering resource in the target rendering resource needs to save rendering data into the hardware graphics processor, based on a mapping relationship between a type of the rendering resource and an access interface, an access interface required to be called is selected for the second rendering resource, the access interface defining access logic for accessing preset data from the hardware graphics processor; and the rendering data required by the second rendering resource to be saved in the hardware graphics processor is written into the hardware graphics processor by using the selected access interface.

17. A virtual machine migration system, comprising a source host where a target virtual machine is located and a destination host to which migration is required: the source host is configured to perform the virtual machine migration method of any one of claims 1-9; 25 the destination host is configured to perform the virtual machine migration method of any one of claims 10-16.

18. A computing device, comprising a memory, a processor and a communication component; the memory is configured to store one or more computer instructions; the processor is coupled with the memory and the communication component, and is configured to execute the one or more computer instructions for performing the virtual machine migration method of any one of claims 1-9 as a source host, or performing the virtual machine migration method of any one of claims 10-16 as a destination host.

19. A computer-readable storage medium storing a computer program, when the computer program is executed by one or more processors, causing the one or more processors to perform the virtual machine migration method of any one of claims 1-16.

20. A computer program product, comprising a computer program, when the computer program is executed by one or more processors, causing the one or more processors to perform the virtual machine migration method of any one of claims 1-16.

Images