A method and system for virtual machine fault migration within a distributed data cloud platform

Abstract

This invention relates to a virtual machine fault migration method and system within a distributed data cloud platform. The method includes the following steps: real-time acquisition of the actual operating status of virtual machines corresponding to each computing node, while simultaneously accessing the theoretical operating status of each virtual machine stored in the metadata server; comparing the actual operating status with the theoretical operating status to determine whether a virtual machine object in a service anomaly state meets the restart conditions; if so, storing the virtual machine object in a fault migration queue, where each virtual machine object in the fault migration queue has a corresponding waiting time; when the waiting time of a virtual machine object in the fault migration queue expires, retrieving the virtual machine object from the fault migration queue, sending a start request, and realizing fault migration. Compared with existing technologies, this invention can simultaneously ensure rapid recovery of virtual machines when the cluster is idle and prioritize the recovery of more important virtual machines when the cluster is congested.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a method and system for virtual machine fault migration within a distributed data cloud platform. Background Technology

[0002] Cloud computing is a means of providing services by treating applications, computing power, storage, and networks as resources. The emergence of various distributed systems has greatly expanded the resource provisioning capabilities of cloud computing. One direction for cloud computing optimization is how to utilize limited resources to provide users with continuous and reasonable services. With the development and maturity of virtualization technology, service providers have shifted from physical environments with poor flexibility and portability to highly flexible virtual machines. Virtual machines can run on various physical environments, facilitating unified management, better ensuring business continuity, and maximizing the utilization of existing hardware and software resources. In current cloud computing services, system architectures based on virtual resource pools are almost indispensable.

[0003] Cloud computing platforms can contain hundreds or even thousands of virtual machines. Therefore, a virtual machine scheduling platform that automates the allocation and management of various virtual machine resources is indispensable. A good virtual machine scheduling platform can improve the efficiency of IT service provision while reducing operational costs and enhancing cloud availability. Like physical machines, virtual machines can also experience unexpected downtime. In such cases, quickly starting redundant or standby servers to restore the virtual machines and their services is part of the high availability characteristics of the cluster. When cluster resources are limited, the services provided by virtual machines vary in importance, but existing virtual machine recovery technologies do not consider this characteristic, making it difficult to guarantee data security. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a virtual machine fault migration method and system in a distributed data cloud platform, which can simultaneously ensure the rapid recovery of virtual machines when the cluster is idle and the priority recovery of more important virtual machines when the cluster is congested.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A virtual machine fault migration method within a distributed data cloud platform, applied to a central node server, wherein the central node server is connected to a metadata server and multiple computing nodes, and each computing node corresponds to multiple virtual machines, includes the following steps:

[0007] The actual running status of the virtual machines corresponding to each computing node is obtained in real time, and the theoretical running status of each virtual machine stored in the metadata server is also retrieved.

[0008] The actual operating state is compared with the theoretical operating state to determine whether the virtual machine object in the service abnormal state meets the restart conditions. If yes, the virtual machine object is stored in the fault migration queue. If no, the virtual machine object is set to the non-started state. Each virtual machine object in the fault migration queue has a corresponding waiting time.

[0009] When the waiting time for a virtual machine object in the failover queue expires, the virtual machine object is retrieved from the failover queue, a startup request is sent, and failover is achieved.

[0010] The waiting time is determined based on the order in which each virtual machine object is stored in the fault migration queue and the priority of that virtual machine object.

[0011] Furthermore, the abnormal service status refers to the actual running status being not running, while the theoretical running status is being started.

[0012] Furthermore, the restart condition is: the current number of retries for the virtual machine object is less than or equal to the maximum number of retries.

[0013] Furthermore, when determining the waiting time, the priorities of the virtual machine objects currently stored in the fault migration queue are compared with those of the virtual machine objects already existing in the fault migration queue. The waiting time of the virtual machine objects with lower priority in the fault migration queue is updated, and the waiting time of the virtual machine objects with lower priority is longer.

[0014] Furthermore, the waiting time for updating low-priority virtual machine objects in the fault migration queue is specifically as follows:

[0015] The waiting time for a low-priority virtual machine object is increased by an interval based on the original waiting time. The interval is determined according to the priority difference between the virtual machine object and the virtual machine object that is jumping the queue.

[0016] Furthermore, the method also includes:

[0017] The waiting time of each virtual machine object in the fault migration queue is detected, and it is determined whether the waiting time is greater than or equal to the starvation time of the virtual machine object. If so, a start request is immediately sent to the corresponding virtual machine object.

[0018] Furthermore, the starvation time is determined by the priority of the virtual machine object.

[0019] Furthermore, when communication with a computing node fails, the computing node is marked as offline, and the real-time running status of all virtual machines corresponding to that computing node is marked as not running.

[0020] The present invention also provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, said one or more programs including instructions for performing the virtual machine failover method within a distributed data cloud platform as described above.

[0021] The present invention also provides a virtual machine failover system within a distributed data cloud platform, comprising one or more processors, a memory, and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the virtual machine failover method within the distributed data cloud platform as described above.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. This invention utilizes virtualization and distributed computing technologies to analyze and compare the real-time status of each virtual machine, promptly restart abnormal virtual machines, and set priorities through reasonable fault migration methods. It also ensures the rapid recovery of virtual machines when the cluster is idle and prioritizes the recovery of more important virtual machines when the cluster is congested, thereby ensuring the high availability and reliability of the system and maximizing the support for high-priority virtual machines to be in service.

[0024] 2. This invention addresses virtual machine failures in various scenarios, such as startup failures, unexpected VM crashes, and compute node offline events, resulting in high reliability.

[0025] 3. This invention sets a starvation time for each virtual machine to prevent low-priority startup time from being too long, thereby further improving reliability. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the structure of the distributed data cloud platform used in this invention;

[0027] Figure 2 This is a schematic diagram of the virtual machine fault migration process of the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0029] Virtual machine failover refers to the process of automatically and quickly resuming the operation of virtual machines on a resource-redundant host machine when one or more virtual machines in a cluster unexpectedly stop. When cluster resources are limited, virtual machines offer services of varying importance. As users, prioritizing the rapid recovery of critical virtual machines is crucial for service availability in the event of an unexpected incident. In this context, a comprehensive failover solution can enable the system to utilize limited resources to recover more critical virtual machines. This invention is proposed for this purpose.

[0030] This invention provides a virtual machine fault migration method within a distributed data cloud platform, which can be applied to resource scheduling optimization scenarios in data cloud platforms with limited resources. When a fault occurs, it can restore high-priority virtual machines as quickly as possible based on virtual machine priority, thereby ensuring the service quality of the data cloud platform.

[0031] The structure of the distributed data cloud platform used in this invention is as follows: Figure 1 As shown, it includes a Master node (central node) containing a RESTful server and a central node management module Master Controller; a metadata server (cluster) storing all metadata information within the cluster; several compute nodes that carry and schedule virtual machine services, including a compute node management module (Compute Manager, CM); and a storage server (cluster) that provides storage for virtual machines (VMs).

[0032] In the above structure, the central node is elected and its functions include providing a user interface, monitoring cluster status, managing slave nodes, and recording persistent virtual machine (VM) metadata (including name, unique identifier, running node, actual running status, theoretical running status, etc.) on a metadata server. It also distributes virtualization layer requests to the compute nodes for completion. The compute node management process runs on all distributed compute nodes, providing resource configuration for VMs, including CPU, memory, and network interface cards. The compute node management module periodically communicates with the VMs running on that compute node, collects VM information, and periodically reports the health information of all VMs running on that compute node to the Master Controller.

[0033] The distributed data cloud platform operates as follows: Users initiate requests to the RESTful server via API. The Master Controller retrieves necessary information from the metadata server and, if needed, distributes the requested content to the CM (Configuration Controller). The CM can then perform corresponding operations on the virtual machines at the virtualization layer. The CM is also responsible for communicating with agents within the VMs to implement internal VM operations, such as network configuration. The Master Controller simultaneously communicates with the storage server to provide storage services to the VMs. The CM acts as the execution unit, managing resource allocation on the host and periodically reporting resource status to the Master Controller. During periodic sampling, the Master Controller marks virtual machines running on the compute node that are not reported by the compute node but are listed as running on that node in the metadata server as requiring failover and adds them to the failover queue. Furthermore, if a compute node loses connection with the metadata server, its node status is marked as offline, and the virtual machine on that node is also considered to require failover during Master Controller sampling and added to the failover queue.

[0034] Specifically, the Compute Manager (CM) is responsible for monitoring the status of the VMs maintained on the compute node. When the compute node management service is normal but a VM crashes, the live VMs reported periodically by the compute node will no longer include that VM. After receiving the information from the CM, the Master Controller will update the actual status of the VMs. If the VM's theoretical running status is "started" but its actual status is "not running," the failover process will continue. When a compute node goes offline or loses connection due to physical failure, network failure, human intervention, or other reasons, the compute node management module will start a scheduled task. If the connection with the Master cannot be restored after a period of time, the node will clean up the virtual machine processes on the node. Similarly, the Master node will mark the compute node as offline and all VMs running on that node as offline, and continue the failover process.

[0035] The VM failure migration marking method is as follows:

[0036] The CM monitors the VM status and reports the basic status periodically. The Master Controller obtains the VM status and then retrieves the VM's theoretical status from the metadata server.

[0037] Table 1

[0038]

[0039] As shown in Table 1, under normal conditions, VM power-on / off transitions between states ② and ③ in the table. State ① in the table represents an illegal state, and state ④ represents a service exception state. Fault migration only handles VMs in state ④.

[0040] To ensure that higher-priority VMs start first when resources are limited, the Master Controller creates a sub-thread for a Priority Queue (PQ). Whenever a VM needs to be migrated due to a failure, the VM instance with its unique identifier and priority is added to the PQ. The PQ sets an initial waiting time for each VM based on its priority and begins a countdown. The initial waiting times for adjacent priority VMs differ by a base interval. After the waiting time expires, the VM will search for a suitable host to start. If a VM of a certain priority joins the PQ while there are already waiting VMs, the waiting time for VMs with lower priorities in the PQ will be increased by a multiple of the base interval, equal to the difference in priority level between the VM and the VM joining the queue. To prevent low-priority VMs from waiting for too long under high cluster load, each VM has a starvation period. Once the starvation period is reached, the VM will immediately search for a suitable host to start.

[0041] VMs that are theoretically online but are actually offline are marked as failover VMs and added to PQ (Power Queuing). Under abnormal shutdown conditions, the actual status of a VM will change from online to offline in two situations: ① The compute node is online and periodically checks for live VM processes. When reporting to the Master node, the Master node compares the information with the VM node distribution structure obtained from the metadata server. Missing VMs will enter the failover process. ② When the Master node loses connection with the compute node, the compute node will clean up the running VM processes after a certain period of time, and the Master node will mark the compute node as offline.

[0042] Each VM has a priority, defined by the user, denoted as p. x When a virtual machine enters the failover queue, it must wait at least T(p) days. x The waiting time is set for virtual machine objects with lower priority, so that higher priority VMs can occupy limited resources earlier under high load conditions.

[0043] A new VM crash event will trigger a delay in the wait queue, specifically: a VM with priority p n After a virtual machine enters the queue, all virtual machines with a priority lower than p... n Virtual machines will start after a certain delay to meet the requirements.

[0044] T = T0 + T(p)m -p n )+T(p m )

[0045] To prevent low-priority VMs from taking too long to start, each VM has a maximum failover wait time, denoted as T, based on its priority. starve (p x When the waiting time T≥T starve (p x When ), the VM will immediately enter the startup process.

[0046] Based on the above technology, this invention provides a virtual machine fault migration method within a distributed data cloud platform, applied to a central node server, i.e., the aforementioned central node, wherein the central node server is connected to a metadata server and multiple computing nodes, and each computing node corresponds to multiple virtual machines, including the following steps:

[0047] 1) Obtain the actual running status of the virtual machines corresponding to each computing node in real time, and at the same time call the theoretical running status of each virtual machine stored in the metadata server;

[0048] 2) Compare the actual running state with the theoretical running state to determine whether the virtual machine object in the service abnormal state meets the restart conditions. If yes, store the virtual machine object in the fault migration queue. If no, set the virtual machine object to the non-started state. Each virtual machine object in the fault migration queue has a corresponding waiting time. The service abnormal state means that the actual running state is non-running and the theoretical running state is started. The restart condition is that the current retry count of the virtual machine object is less than or equal to the maximum retry count.

[0049] 3) When the waiting time for a virtual machine object in the fault migration queue expires, the virtual machine object is retrieved from the fault migration queue, a startup request is sent, and fault migration is achieved.

[0050] The waiting time is determined based on the order in which each virtual machine object is stored in the fault migration queue and the priority of that virtual machine object.

[0051] The lower the priority of a virtual machine object, the longer its waiting time. When determining the waiting time, the priorities of virtual machine objects currently stored in the failover queue are compared with those already existing in the failover queue. The waiting time of lower-priority virtual machine objects in the failover queue is updated, specifically by adding an interval time to the original waiting time. This interval time is determined based on the priority difference between the virtual machine object and any other virtual machine object that has jumped the queue.

[0052] In another preferred embodiment, the method further includes: detecting the waiting time of each virtual machine object in the fault migration queue, determining whether the waiting time is greater than or equal to the starvation time of the virtual machine object, and if so, immediately sending a start request to the corresponding virtual machine object. The starvation time is determined by the priority of the virtual machine object.

[0053] In another preferred embodiment, when communication with a computing node fails, the computing node is marked as offline, and the real-time running status of all virtual machines corresponding to the computing node is marked as not running.

[0054] Example

[0055] like Figure 2 As shown in this embodiment, the virtual machine fault migration process under the distributed data cloud platform is described as follows:

[0056] When a VM starts normally, the Master Controller registers its theoretical running state as running status on the metadata server. During runtime, the CM (Compute Manager) periodically reports the VM's service status. In cases of startup failure, unexpected VM crashes, or compute node offline events, the Master Controller, upon sampling, detects a discrepancy between the VM's theoretical and actual running states and marks the VM as requiring failover. Each VM has a priority, indicating its importance. If the maximum number of failover retries has not been reached, the VM enters the failover queue (PQ); otherwise, the VM failover fails and returns to the inactive state. Upon entering the PQ, the VM receives a base waiting time based on its priority; higher-priority VMs have shorter waiting times. Additionally, when a VM enters the queue, if an existing instance has a lower priority, an extra time is added to its current waiting time, equivalent to the difference between the base waiting times of the two VMs. After entering the queue, the VM waits according to its base waiting time. If a new, higher-priority VM joins during this waiting period, the waiting time is extended according to the same rules. To prevent the continuous addition of new VMs when the cluster is unstable, which could significantly delay the recovery of low-priority VMs, each VM instance is assigned a starvation period attribute based on its priority. If the total waiting time reaches the starvation period, the Master Controller will immediately attempt to start the VM instance, regardless of the remaining waiting time. After the waiting period ends, the VM will leave the PQ (Process Queuing), and the Master Controller will issue a VM start request.

[0057] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for virtual machine fault migration within a distributed data cloud platform, characterized in that, This method is applied in a central node server, which connects to a metadata server and multiple compute nodes, each compute node corresponding to multiple virtual machines, and includes the following steps: The actual running status of the virtual machines corresponding to each computing node is obtained in real time, and the theoretical running status of each virtual machine stored in the metadata server is also retrieved. The actual operating state is compared with the theoretical operating state to determine whether the virtual machine object in the service abnormal state meets the restart conditions. If yes, the virtual machine object is stored in the fault migration queue. If no, the virtual machine object is set to the non-started state. Each virtual machine object in the fault migration queue has a corresponding waiting time. When the waiting time for a virtual machine object in the failover queue expires, the virtual machine object is retrieved from the failover queue, a startup request is sent, and failover is achieved. The waiting time is determined based on the order in which each virtual machine object is stored in the fault migration queue and the priority of that virtual machine object. When determining the waiting time, the priorities of the virtual machine objects currently stored in the fault migration queue are compared with those of the virtual machine objects already existing in the fault migration queue. The waiting time of the virtual machine objects with lower priority in the fault migration queue is updated, and the waiting time of the virtual machine objects with lower priority is longer. The waiting time for low-priority virtual machine objects in the update fault migration queue is specifically as follows: The waiting time for a low-priority virtual machine object is increased by an interval based on the original waiting time, and the interval is determined according to the priority difference between the virtual machine object and the virtual machine object that is jumping the queue. The method also includes: The waiting time of each virtual machine object in the fault migration queue is detected, and it is determined whether the waiting time is greater than or equal to the starvation time of the virtual machine object. If so, a start request is immediately sent to the corresponding virtual machine object. The starvation time is determined by the priority of the virtual machine object.

2. The virtual machine fault migration method in a distributed data cloud platform according to claim 1, characterized in that, The service abnormal state refers to the actual running state being not running, while the theoretical running state is being started.

3. The virtual machine fault migration method in a distributed data cloud platform according to claim 1, characterized in that, The restart condition is: the current number of retries for the virtual machine object is less than or equal to the maximum number of retries.

4. The virtual machine fault migration method in a distributed data cloud platform according to claim 1, characterized in that, When communication with a compute node fails, the compute node is marked as offline, and the real-time running status of all virtual machines corresponding to that compute node is marked as not running.

5. A computer-readable storage medium, characterized in that, It includes one or more programs that are executed by one or more processors of an electronic device, the one or more programs including instructions for performing a virtual machine failover method within a distributed data cloud platform as described in any one of claims 1-4.

6. A virtual machine failover system within a distributed data cloud platform, characterized in that, It includes one or more processors, memory, and one or more programs stored in the memory, said one or more programs including instructions for performing the virtual machine failover method within the distributed data cloud platform as described in any one of claims 1-4.

Images