Generating a relocation log for virtual machines

Abstract

A computer-implemented method comprising: Obtaining (275), by a first controller (211), first power-related information from a first power system group (201) comprising one or more first power systems that provide power to a first group (200) of powered hardware components associated with the first controller, the first group of powered hardware components executing a first plurality of virtual machines (212); and Generating (280), by the first controller (211), a relocation protocol to migrate the first plurality of virtual machines (212) based at least in part on the first power-related information, the relocation protocol including: a migration (281) of a first subset of one or more virtual machines of the first plurality of virtual machines (212) such that the first subset of one or more virtual machines is migrated to a second group (220) of powered hardware components and is performed such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration; and creating a snapshot (282) of a second subset of one or more virtual machines of the first plurality of virtual machines (212).

Description

BACKGROUND

[0001] As is known, a virtual machine (VM) is a software implementation of a machine (e.g., a computer) that executes computer programs. A VM typically emulates a physical computing environment with requirements for a central processing unit (CPU), memory, disk, and other hardware resources, managed by a virtualization layer that translates these requirements to the underlying physical hardware. Virtual machines are created within a virtualization layer, such as a hypervisor or a virtualization platform, that runs on top of a client or server operating system. System migrations of virtual machines can be desirable for various reasons, including the ability to provide a backup system while hardware and / or software updates are installed.System migrations can also be performed to move a virtual machine to a new processor or different hardware. There are several approaches to performing the migration, including pausing a virtual machine while it is running, as well as performing a live migration, which migrates a virtual machine while it is running. Live migration of a virtual machine is often desired by companies with mission-critical systems.

[0002] Power outages in data centers can occur for several reasons, including natural disasters. Natural disasters or other weather-related causes have become an increasingly common variable in data center availability. To ensure that a data center, including all virtual machines running within it, can continue running during a utility power outage, battery backup systems and gas-powered generators can be used to continue supplying power to the data center, for example, to enable an orderly shutdown and / or backup of running systems.However, in many cases, data may ultimately be lost due to ineffective backup mechanisms or backup mechanisms that never engaged or were not completed before battery backup systems ran out of power or backup generators ran out of fuel.

[0003] The document DE 10 2014 114 108 A1 describes a process control system and method. An exemplary method includes operating a first cluster with first virtual machines and first servers, and operating a second cluster with second virtual machines and second servers. The exemplary method further comprises storing first data of the first virtual machine in a first data store of the first cluster and storing a replica of the first data in a second data store of the second cluster. The exemplary method further comprises storing second data of the second virtual machines in the second data store and storing a replica of the second data in the first data store, as well as identifying a failure of the first cluster.The method also includes, in response to the failure, restarting the first virtual machines using the second servers and replicating the first data in the second datastore.

[0004] Document US 2012 / 0102492 A1 describes a power supply for supplying power to a computer, comprising an instruction acquisition unit configured to acquire an identifier of a virtual machine and an instruction for the virtual machine, the virtual machine being executed in emulation by a virtual host executing on the computer; and a virtual machine management unit configured to input an operation instruction for the virtual machine into the virtual machine based on the acquisition by the instruction acquisition unit.

[0005] Document US 2016 / 0349820 A1 describes a method comprising monitoring energy consumption parameters in a data center on a controller and automatically modifying components within the data center to reduce energy consumption based on the energy consumption parameters. The energy consumption parameters include the routing paths between the components within the data center. A device and logic are also disclosed herein.

[0006] The document US 2013 / 0111492 A1 describes an energy-saving control method for an information processing system including a group of information processing devices with a plurality of types of interconnected information processing devices and cooling devices that cool the group of information processing devices. SUMMARY OF THE INVENTION

[0007] Deficiencies of the prior art are overcome and advantages are provided by providing a computer-implemented method that includes obtaining first power-related information from a first power system group that includes one or more first power systems that provide power to a first group of powered hardware components associated with the first controller. The first group of powered hardware components executes a first plurality of virtual machines. The method also includes generating, by the first controller, a relocation log to migrate the first plurality of virtual machines based at least in part on the first power-related information.The relocation protocol includes: migrating a first subset of one or more virtual machines of the first plurality of virtual machines such that the first subset of one or more virtual machines is migrated to execute on a second set of powered hardware components such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration; and taking a snapshot of a second subset of one or more virtual machines of the first plurality of virtual machines.

[0008] Advantageously, in one or more aspects, a computer-implemented method, system, and computer program product are provided for enabling dynamic generation of a relocation protocol for virtual machines based at least in part on power-related information obtained, for example, from a first power system group that provides power to a first group of powered hardware components on which the virtual machines are executed. The relocation protocol for migrating the virtual machines may include: migrating a first subset of one or more virtual machines such that the first subset of one or more virtual machines is migrated to execute on a second group of powered hardware components such that the first subset of one or more virtual machines is migrated to execute on amultiple virtual machines continue to operate in a substantially continuous manner during the migration; and taking a snapshot of a second subset of one or more virtual machines of the virtual machines executing on the first set of hardware components. By dynamically determining the relocation protocol, processing can better prioritize the migration of virtual machines to ensure a more efficient transfer of the virtual machines from, for example, the first set of powered hardware components to the second set of powered hardware components.This relocation protocol may seek to minimize or mitigate data loss resulting, for example, from a natural disaster occurring near the first group of powered hardware components, causing an impending power outage to the first group of powered hardware components.

[0009] As further explained herein, generating the relocation log may include referencing historical data related to the migration of a virtual machine or a snapshot creation of a virtual machine to place one or more of the virtual machines into either the first or second subset of virtual machines. For example, the historical data may include information about the length of time required for a live migration or snapshot creation of a virtual machine of a particular type, workload size, etc.

[0010] Additionally or alternatively, the relocation log may be generated using associated priority information from one or more of the virtual machines to be migrated. For example, in one or more embodiments, a customer may pay for higher priority of service in the event of a power outage, and thus the one or more virtual machines running the client's workload may be identified for live migration. Further, generating the relocation log may consider the power status of intermediate hardware components and data paths between, for example, a first data center and a second data center, and may even include identifying the particular data center or second group of powered hardware components to which the virtual machines are to be migrated, as well as the data path to be used.

[0011] In one or more implementations, the computer-implemented method further includes: migrating the first subset of one or more virtual machines to the second group of powered hardware components in a manner such that the first group of one or more virtual machines continues to operate in a substantially continuous manner during the migration; and migrating the second subset of one or more virtual machines at least in part by providing the second subset of one or more virtual machines to the second group of powered hardware components based on one or more respective snapshots of the second subset of one or more virtual machines obtained at the first group of powered hardware components. For example, migrating the second subset of one or more virtual machines maymultiple virtual machines after the first subset of one or more virtual machines has been migrated.

[0012] In one or more embodiments, migrating the second subset of one or more virtual machines may include first transferring the respective snapshot(s) of the virtual machine(s) of the second subset of one or more virtual machines to an intermediate group of powered hardware components powered by an intermediate power system group, i.e., before provisioning the second subset of one or more virtual machines to the second group of one or more powered hardware components based on the respective snapshot(s). In such embodiments, the intermediate power system group is different from the first power system group, and transferring the respective snapshot(s) of the virtual machine(s) of the second subset of one or more virtual machines to an intermediate group of powered hardware components powered by an intermediate power system groupmultiple virtual machines to the intermediate group of powered hardware components may occur substantially concurrently with the migration of the first subgroup of one or more virtual machines to the second group of powered hardware components.

[0013] In one or more implementations, the first group of powered hardware components belongs to a first cloud, and the first controller is a first cloud controller, and the second group of powered hardware belongs to a second cloud. The second group of hardware components may be powered by a second power system group that includes one or more second power systems. In one or more embodiments, the first cloud and the second cloud may be hosted in different, geographically separated data centers.

[0014] Further, in one or more implementations, the second cloud may include a second cloud controller, and generating the relocation log by the first controller may include obtaining, by the first cloud controller, second power-related information from the second cloud controller or the second power system group that is indicative of the power state for the second group of powered hardware components.

[0015] In one or more embodiments, generating the relocation log by the first controller may include referencing historical data related to at least one virtual machine migration or virtual machine snapshot creation to enable placing at least one virtual machine of the first plurality of virtual machines in either the first subset of one or more virtual machines or the second subset of one or more virtual machines.

[0016] Additionally, in one or more embodiments, generating the relocation log by the first controller may include using associated priority information of at least one virtual machine of the first plurality of virtual machines and placing the at least one virtual machine in either the first subset of one or more virtual machines or the second subset of one or more virtual machines.

[0017] In one or more embodiments, generating the relocation protocol by the first controller may include evaluating a data transmission path from the first group of powered hardware components to the second group of powered hardware components. Evaluating the data transmission path may include determining the power state for one or more switches in the data transmission path, for example, to confirm continued availability of the data transmission path.

[0018] For example, the first power-related information may include an estimated amount of time until a power failure for the first group of powered hardware components, and the relocation protocol may be an emergency relocation protocol dynamically generated based at least in part on the amount of time until a power failure of the first group of powered hardware components.

[0019] Systems and computer program products relating to one or more aspects are also described and claimed herein. Furthermore, services relating to one or more embodiments may also be described and claimed herein.

[0020] Further features and advantages are realized by the techniques of the present invention. Further embodiments and aspects of the invention are described in detail herein and are considered part of the claimed invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] One or more aspects of the present invention are particularly pointed out and distinctly claimed by way of example in the claims at the end of the specification. The foregoing and other objects, features, and advantages of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: Fig. 1 illustrates an example of a virtual computing environment including virtual machines that can traverse a relocation protocol in accordance with one or more aspects of the present invention; Fig. 2A illustrates an embodiment of a transfer of virtual machines from a first group of powered hardware components to a second group of powered hardware components using a generated relocation protocol in accordance with one or more aspects of the present invention; Fig. 2B illustrates an embodiment of a process for generating a relocation protocol for migrating a first plurality of virtual machines from, for example, the first group of powered hardware components to the second group of powered hardware components of Fig. 2A according to one or more aspects of the present invention; Fig. 3A illustrates another embodiment of a transfer of virtual machines from a first group of powered hardware components or a source system to a second group of powered hardware components or a target system using a generated relocation log including a live migration of one or more virtual machines and a snapshot of one or more other virtual machines, in accordance with one or more aspects of the present invention; Fig. 3B and Fig. 3C illustrates an embodiment of a process for generating a relocation log for live migration of one or more virtual machines from the source system to the target system of Fig. 3A and for snapshotting one or more other virtual machines of the source system for subsequent deployment to the target system in accordance with one or more aspects of the present invention; Fig. 4A illustrates another example of a computing environment that may embody and utilize one or more aspects of the present invention; Fig. 4B further details of the memory of Fig. 4A according to one or more aspects of the present invention; Fig. 5 illustrates another embodiment of a data processing system that may include relocation processing for a virtual machine, in accordance with one or more aspects of the present invention; Fig. 6 illustrates one embodiment of a cloud computing environment that may enable implementation or use in conjunction with one or more aspects of the present invention; and Fig. 7 illustrates an example of extraction model layers that may enable implementation of virtual machine relocation in accordance with one or more aspects of the present invention. DETAILED DESCRIPTION

[0022] Previously, virtual machines could be migrated from one data center to another as a disaster mitigation solution. For example, the migrated data might be secondary backup data that was migrated. However, such data is not as current as the running virtual machines in the primary storage location. Furthermore, there would typically be no way to determine whether the second data center location was also affected by the disaster that compromised data at the primary source location.

[0023] Therefore, in one or more aspects, disclosed herein is an efficient use of correlating available runtime power, for example, at a source system for a first cloud with a time required to offload everything from the cloud to avoid system downtime. In particular, disclosed herein are various aspects approaches for a cloud manager to use information provided, for example, by a data center's disaster recovery application to take appropriate action for its virtual machines. For example, in one or more embodiments, data center disaster recovery information and historical information about virtual machines may be used to determine which action is appropriate for each virtual machine, such as live migration, snapshot creation, or even powering off a particular virtual machine.This may, for example, include using historical information about the time required for each of the possible actions for a particular type of virtual machine or workload size. This may include, for example, the type of application running on the virtual machine, the type of virtual machine, and the hardware on which the virtual machine is running. In one or more further aspects, considering using historical information about the times required to transfer the results of each action. For example, determining the time to create and send a snapshot versus the time for a live migration of a particular virtual machine could be done. This could also include considering available information about the transfer path, for example, from the first data center to the second data center.Furthermore, generating the relocation protocol may include selecting the second data center, for example, based on power availability for the second data center. This may include, for example, exchanging data with a second controller of the second data center and / or a second power system group that supplies power to the second data center to determine power availability at the second data center. Furthermore, this may include considering data center disaster recovery information about participants (e.g., switches or other hardware) in the transmission path.For example, is there sufficient power and thus time on the data transfer path to allow migration or transfer to be performed? Historical information about the performance impact of the recovery state can also be considered when generating the relocation log.

[0024] An analytics approach can be used to determine, for example, the amount of time remaining on a backup generator during a disaster scenario before the backup generator fails, for example, by analyzing historical information. This analysis can include, for example, evaluating the amount of gas in a generator and determining that the amount of gas can last a certain number of hours before the system completely loses power.

[0025] This discloses the concept of considering situations where there is not enough time for a live migration of all virtual machines and instead generating a protocol to create snapshots of selected virtual machines to ensure that nothing is lost, thereby protecting the current state of the virtual machines from being lost in the event that there is not enough power to the source data center or resources available for a live migration of all virtual machines.Analyses that consider historical migration times from one virtual machine to another virtual machine could be used to determine whether there is time to perform the migration. In situations where generator power would deplete before the migration could be completed, a preemptive snapshot of the virtual machine could be created to prevent data loss. The decision to create a snapshot could be made by an administrator, or it could be set up to be processed automatically, for example, in cases where the administrator is unavailable. This way, a new virtual machine could be booted to exactly the state it was in before the power outage, without any data loss.

[0026] Thus, according to one or more aspects, a relocation protocol, such as an emergency relocation protocol, is dynamically created for transferring virtual machines from, for example, a first group of powered hardware components (such as a first (or source) system or data center) to a second group of powered hardware components (such as a second (or destination) system or data center). The relocation protocol, in one or more embodiments, may utilize both live migration of a first subset of one or more virtual machines of the virtual machines to be transferred and snapshot creation of a second subset of one or more virtual machines of the virtual machines to be transferred.The relocation log may be dynamically generated based at least in part on power-related information obtained from a first power system group of one or more first power systems that provide power to, for example, the first group of powered hardware components executing on the plurality of virtual machines.

[0027] In certain embodiments, the first group of hardware components belongs to a first cloud, the first controller is a first cloud controller, and the second group of powered hardware components belongs to a second cloud. The second group of powered hardware components is powered by a second power system group that includes one or more second power systems, where the second power system group may be different from the first power system group. For example, the first cloud and the second cloud may be hosted in different, geographically separated data centers. Further, as described below, one or more intermediate systems, such as one or more intermediate groups of powered hardware components, may be used to enable simultaneous live migration for the first subset of one or more second power systems.multiple virtual machines and to enable the transfer of the respective snapshot(s) of the second subset of one or more virtual machines, for example via different data paths from the first group of powered hardware components.

[0028] In one or more embodiments, the first power-related information includes an estimated amount of time until power failure for the first group of powered hardware components, and the relocation log is an emergency relocation log dynamically generated based at least in part on the amount of time until power failure of the first group of powered hardware components. For example, generating the relocation log by the first controller may include referencing historical data related to, for example, a live migration of one or more virtual machines and / or a snapshot creation of one or more virtual machines.For example, the history data may contain information about the amount of time required for a live migration of a virtual machine of a particular type or workload size, as well as the amount of time required for a snapshot of a virtual machine of a particular type or workload. Based on this information, a generator can dynamically determine an appropriate migration protocol, for example, based on the amount of time remaining until the first group of powered hardware components loses power.

[0029] Alternatively or additionally, generating the relocation protocol by the first controller may include using or referencing associated priority information from one or more virtual machines of the first plurality of virtual machines and placing the one or more virtual machines in either the first subset of one or more virtual machines or the second subset of one or more virtual machines. For example, clients at different priority levels may be eligible to participate should an emergency relocation of their workload become necessary.For example, a client with Gold-level entitlement can ensure that the virtual machines running its workload are live migrated from the first set of powered hardware components to the second set of powered hardware components, at least with priority over clients with lower entitlement levels. In such an implementation, a snapshot can be created for the virtual machines with lower entitlement levels for subsequent deployment to the second set of powered hardware components, for example, based on the estimated amount of time until power failure.

[0030] In particular, some embodiments of the present invention may include one or more of the following features, characteristics, operations, and / or advantages: (i) managing, by a first cloud controller, a first group of one or more power systems that provide power to a first group of one or more powered hardware components of a first cloud, the first group of one or more powered components executing a first plurality of virtual machines (VMs); (ii) receiving, by the first cloud controller and from a first power system of the first group of one or more powered hardware components, power from the first group of one or more powered hardware components.a plurality of powered systems of a first piece of power-related information; (iii) determining, by the first cloud controller, a procedure for securely migrating the first plurality of VMs based at least in part on the first piece of power-related information; (iv) wherein determining the procedure for securely migrating the first group of VMs includes: (a) migrating a first subset of VM(s) of the first plurality of VMs such that the first subset of VM(s) running on a second group of one or more powered systemsa plurality of powered hardware components such that the first subset of VM(s) continues to operate in a substantially continuous manner during the migration; and (b) snapshotting a second subset of VM(s) of the first plurality of VM(s); (v) migrating the first subset of VM(s) of the first plurality of VMs such that the first subset of VM(s) runs on a second group of powered hardware components such that the first group of VM(s) continues to operate in a substantially continuous manner during the migration; (vi) migrating the second group of VM(s) by making them available to the second group of one or more powered hardware components.multiple powered hardware components based on their respective snapshots; (vii) the migration of the second group of VM(s) can be performed after the migration of the first group of VM(s) is complete; (viii) the process does not require adding networks to offload work to another VM; (ix) the process examines the other VMs available on a network and migrates the work to the optimal storage location; (x) the process preemptively creates snapshots so that no data replication is necessary; and / or (xi) the process responds during a disaster to prevent a need for data recovery.

[0031] An example of a computing environment for incorporating and using one or more aspects of a virtual machine relocation capability as described herein is first described with reference to Fig. 1. With reference to Fig. 1, a data processing environment 100 may be implemented in an example on the International Business Machines (IBM) ® ) Corporation, Armonk, New York. The z / Architecture is described in an IBM publication entitled "z / Architecture Principles of Operation," Publication No. SA22-7832-10, Eleventh Edition, March 2015, which is incorporated herein by reference in its entirety. Z / ARCHITECTURE, IBM, Z / VM, and Z / OS (referenced herein) are registered trademarks of International Business Machines Corporation, Armonk, New York, USA. Other names used herein may be registered trademarks, trademarks, or product names of International Business Machines Corporation or other companies.

[0032] In another example, the computing environment may be based on the Power Architecture offered by International Business Machines Corporation, Armonk, New York. One embodiment of the Power Architecture is described in "Power ISA™ Version 2.07B," International Business Machines Corporation, April 9, 2015, which is incorporated herein by reference in its entirety. POWER ARCHITECTURE is a registered trademark of International Business Machines Corporation, Armonk, New York, USA.

[0033] In yet another example, the computing environment may be based on other architectures offered by International Business Machines Corporation, Armonk, New York.

[0034] The computing environment 100 includes a central processor complex (CPC) 102 that provides support for virtual machines. The CPC 102 is connected to one or more input / output (I / O) units 106 via one or more control units 108. The central processor complex 102 includes, for example, a processor memory 104 (also known as main memory, main storage, central memory) connected to one or more central processors (also known as central processing units (CPUs)) 110 and an input / output subsystem 111, each of which is described below.

[0035] The processor memory 104 contains, for example, one or more virtual machines 112, a VM manager such as a hypervisor 114 that manages the virtual machines, and a processor firmware 115. An example of the hypervisor 114 is z / VM ®offered by International Business Machines Corporation, Armonk, New York. The hypervisor is sometimes referred to as the host. As used herein, firmware further includes, for example, the microcode and / or millicode of the processor. It includes, for example, the hardware-level instructions and / or data structures used in an implementation of higher-level machine code. In one embodiment, it includes, for example, proprietary code, typically provided as microcode, that includes trusted software or microcode specific to the underlying hardware that controls operating system access to the system hardware.

[0036] The CPC's virtual machine support provides the ability to run a large number of virtual machines 112, each capable of running different programs 122 and a guest operating system 120, such as Linux. Each virtual machine 112 is capable of operating as a separate system. That is, each virtual machine can be reset independently, run a guest operating system, and work with different programs. An operating system or application program running in a virtual machine may appear to have access to an entire and complete system, while in reality, only a portion of it is available.

[0037] Processor memory 104 is connected to central processors (CPUs) 110, which are physical processor resources assignable to virtual machines. For example, virtual machine 112 includes one or more logical processors, each of which represents all or a portion of a physical processor resource 110 that can be dynamically assigned to the virtual machine.

[0038] Furthermore, the processor memory 104 is connected to an I / O subsystem 111. The input / output subsystem 111 controls the flow of information between the input / output controllers 108 and devices 106 and the main memory 104. It is connected to the central processor complex in such a way that it may be part of the central processor complex or separate from it.

[0039] For example, one or more data processing environments, such as in Fig. 1, may be present in a data center, and as noted, power outages in a data center can occur for a variety of reasons, including natural disasters. To ensure that a data center, including any virtual machines running on systems within a data center, continues to operate during a utility power outage, battery backup systems and / or gas-powered generators may be used to continue to supply power to the data center for at least a period of time, for example, to enable an orderly shutdown and / or backup of running systems. In the event of a natural disaster, the main power supply may be out for several days or even weeks.Therefore, in one or more aspects, a process is disclosed herein for dynamically generating a relocation log to relocate virtual machines from one data center, for example, to another, remotely located data center to enable continuous operation of the virtual machines.

[0040] Fig. 2A illustrates one embodiment of a process for transferring virtual machines from a first group of powered hardware components 200, such as a source system or source data center, to a second group of powered hardware components 220, such as a target system or target data center. The first group of powered hardware components 200 is powered by a first power system group 201 from one or more first power systems, and the second group of powered hardware components 220 is powered by a second power system group 221 from one or more second power systems. In practice, the first and second power system groups 201, 221 are at least partially different power system groups.As shown, the first group of powered hardware components 200 may include a first cloud environment 210, and the second group of powered hardware components 220 may include a second cloud environment 230. The first cloud 210 may have a first cloud controller 211, and the second cloud 230 may have a second cloud controller 231. As illustrated, the first cloud 210 may also include a plurality of virtual machines 212 executing on the first group of powered hardware components 200.

[0041] As disclosed herein, a controller, such as the first cloud controller 211, may be provided with the ability to obtain and utilize data from one or more power systems, such as the first power system group 201 and the second power system group 221, to dynamically determine or generate a relocation protocol containing where and how to safely relocate the virtual machines of the associated system, such as the first group of powered hardware components 220. This may be particularly useful if a natural disaster has occurred resulting in the loss of the primary power supply to the first group of powered hardware components 200, and only a limited backup time remains.

[0042] As disclosed herein, the controller is provided with a capability or ability to generate a relocation protocol for migrating the plurality of virtual machines 212 based at least in part on power-related information (e.g., first power-related information) obtained from at least the first power system group 201. For example, the power-related information may include an amount of time remaining before power is lost for the first group of powered hardware components 200. The relocation protocol may enable migration of a first subset of one or more virtual machines (e.g., virtual machine 1 in Fig. 2A) of the plurality of virtual machines 212, such that the first group of one or more virtual machines is migrated to and executed on the second group of powered hardware components 220 such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration. This migration refers to live migration, which is a process of moving a running virtual machine or application between different physical machines without disconnecting the client or application. Memory, storage, and network connectivity of the virtual machine are transferred from an original machine (e.g., the first group of powered hardware components 200) to a destination or target machine (e.g., a second group of powered hardware components 220).Pre-copy or post-copy memory migration can be used, and various VM managers provide live migration support, as well as various cloud platforms and systems known in the field. In a specific example, IBM provides ® POWER Hypervisor™ provides live migration support.

[0043] In addition, the generated relocation log contains a snapshot creation of a second subset of one or more virtual machines (e.g., virtual machine 2, virtual machine 3 in Fig. 2A) of the majority of virtual machines 212. As is known, a snapshot saves the state and data of a virtual machine at a specific point in time. The state contains the power state of the virtual machine (for example, powered on, powered off, or suspended). The data includes all the files that comprise the virtual machine. This includes disks, memory, and other devices such as virtual network interface cards. A virtual machine provides several options for creating and managing snapshots and snapshot chains. These operations allow the controller to create snapshots, revert to any snapshot in the chain, and remove snapshots. Snapshot creation is currently supported by various VM managers, for example, various machines from VMware, Inc.Additionally, snapshot creation is supported by several IBM products, including IBM Virtual Servers™ and IBM Cloud Manager™.

[0044] In general, the time required to create and transfer a snapshot from a source system to a target or destination system may be considerably shorter than the time required to live migrate a virtual machine from the source system to the destination system. Thus, the power-related information from the first group of one or more power systems 201, such as the remaining available time for the first group of powered hardware components 200, may be used by the cloud controller 211 to generate the relocation log to ensure that substantially all virtual machines are migrated to the second group of powered hardware components or have a snapshot created for transfer to and subsequent use by the second group of hardware components.

[0045] Fig. 2B illustrates one embodiment of a process for dynamically generating a relocation log and then transferring or migrating virtual machines, for example, during a disaster where virtual machines need to be transferred from one data center to another, remotely located data center.

[0046] With reference to Fig. 2B, the process includes managing, by a first cloud controller, a first power system group that provides power to a first group of powered hardware components of a first cloud, wherein a first group of powered hardware components executes a first plurality of virtual machines, 270. The first cloud controller receives or obtains first power-related information from a first power system of the first power system group, 275. As noted, this information may include, for example, an amount of time for remaining available power to the first group of powered hardware components.

[0047] The first cloud controller then generates a location and a protocol to migrate the first plurality of virtual machines based at least in part on the power-related information, 280. The move protocol may include: migrating a first subset of one or more virtual machines of the first plurality of virtual machines such that the first subset of one or more virtual machines executes on a second set of powered hardware components such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration, 281; and preemptively snapshotting a second subset of one or more virtual machines of the first plurality of virtual machines, 282. The processing may further comprise migrating the first subset of one or morea plurality of virtual machines of the first plurality of virtual machines, such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration, 285. Furthermore, the second subset of one or more virtual machines may be migrated based on their respective snapshots (for example, by making the second subset of one or more virtual machines available to the second group of powered hardware components after the migration of the first subset of one or more virtual machines is complete), 290.

[0048] In one or more aspects, the processing disclosed herein relies on a global migratory view, where the cloud controller works with other systems, such as other cloud controllers and other power systems, to investigate the availability and performance of the systems for migrating the virtual machines to which it is connected. For example, in cases where workloads are running in a cloud environment that may be down, for example, due to a power outage, virtual machines may be migrated to a target or destination cloud that is not down. In such a case, migration may require decommissioning the virtual machines to be moved to the new target cloud and transferring data (memory, processing state, etc.) to the target cloud.This work can be prioritized from a bandwidth perspective to avoid a power outage occurring while the virtual machines are running. In parallel, for virtual machines for which live migration to the target environment is not possible, a snapshot can be created. This captures their state and saves it, for example, to disk in a way that does not affect the migration of the active virtual machines. Along with the snapshot creation, these virtual machines would be shut down until a subsequent deployment to the target system.

[0049] In one or more embodiments, the controller, such as the cloud controller discussed above, may weigh a number of factors in generating or determining a particular relocation protocol for the virtual machines to which they belong. These factors may include determining what actions to take instead of defaulting to a migration protocol of pure migration and shutdown, including the option of taking a snapshot of one or more of the running virtual machines for transfer. The controller selects or determines which of the virtual machines will be migrated and for which a snapshot will instead be taken, with the snapshot being transferred to the target or destination system.For example, this could include evaluating the history of snapshot creation of a virtual machine of a particular type, the resulting snapshot size, and the available bandwidth for transferring the snapshot. Furthermore, the cloud controller could investigate whether the snapshot can be transferred to an intermediate system or node, for example, using data paths that would use bandwidth not required for migrating the other virtual machines. In this way, the transferred snapshots can subsequently be transferred to the target or destination system for recovery or deployment. Furthermore, the cloud controller can ensure that all transfer nodes (since multiple nodes could be used) do not offload themselves by exchanging data with the cloud controller of the transfer node.If they themselves are also offloaded, the controller would search for alternative nodes, such as alternative intermediate nodes, and if none are available, may negotiate with one or more cloud controllers to determine whether operation will last long enough to be used as a transmission node. For intermediate hardware, such as intermediate switches to be used in transmission (i.e., switches in the data path), the cloud controller may work with a cloud controller to which the intermediate hardware belongs, or, if independent, may communicate directly with the power system belonging to the intermediate hardware.While the migration of one or more virtual machines is in progress, a snapshot can be created in parallel for the one or more virtual machines for which a snapshot is to be created and then transferred to the target or destination node, system, data center, etc.

[0050] Fig. 3A illustrates a more detailed example of a virtual machine migration, for example, from a source system 300 to a target system 330, which may utilize, in part, one or more intermediate systems 350. In this embodiment, a source system 300, such as a first group of powered hardware components, includes a first cloud 310 with a plurality of virtual machines 312 running and having an associated controller 311. The source system 300 is powered by a first power system group (Power System(s) A 301) including one or more power systems.The migration of the virtual machines 312 is to be performed over a network 320, which may include one or more intermediate hardware components, such as switches 322, 326, each of which is powered by an associated power system group (such as power system(s) B 324, power system(s) C 328), each of which may include one or more power systems.

[0051] In this example, the target system 330, such as a second group of powered hardware components, may host a second cloud environment 340 having an associated second controller 341. The target system 330 may also be powered by a second power system group (e.g., power system(s) D 331) from one or more power systems. As noted, one or more intermediate systems 350 may be used to enable, for example, the concurrent transfer of snapshot data at the time one or more virtual machines are migrated, for example, across different data paths 321, 325. The intermediate system 350, in one or more examples, may include memory 360 for receiving snapshots 362 of virtual machines and an associated controller 361. The intermediate system(s) 350 mayThe intermediate systems 350 may also be powered by an intermediate power system group (Power System(s) E 351), which may include one or more power systems. In general, the power system groups, such as Power System(s) A 301, Power System(s) B 324, Power System(s) C 328, Power System(s) D 331, and Power System(s) E 351, may include or be different groups of power supply groups, or may, in one or more other embodiments, have the same power system(s), such as Power System(s) B 324 and Power System(s) C 328 of the network 320.

[0052] Fig. 3B and Fig. 3C illustrate an embodiment of a process for migrating virtual machines according to one or more aspects of the present invention. With joint reference to Fig. 3A to 3C, a power failure notification may be received from the power system(s) A 301 to the source system 370, such as at the first cloud controller 311. Based on this, the first cloud controller or manager determines where the virtual machines should be migrated, i.e., it determines the target or destination system (or the second group of powered hardware components) 372.

[0053] In one or more embodiments, the cloud controller manager 311 may query power systems, such as the power system(s) B 324 that power a switch 1 322 in the data path 321 between the source system and the target system 374. In addition, the cloud controller or manager 331 of the source system 300 may determine the status of the power system(s) D 331 that power the target system 376. This may include the first cloud controller 311 querying the second cloud controller 341 of the target system 330, or it may include the cloud controller 311 communicating directly with the power system(s) D 331 to obtain the desired information.

[0054] Next, in one or more embodiments, the cloud controller 311 may determine 378 one or more intermediate systems or nodes capable of receiving snapshot data, as well as the state of the power system(s) E 351 for the intermediate system(s). Along with this information, the cloud controller or manager may determine 380 the state of the power system(s) C 328 that power one or more switches 326 in a data path 325 between the source system and the intermediate system(s).Along with this information, further information is determined, for example, historical information about the time required for a virtual machine migration or a virtual machine snapshot creation, or priority data in which one or more virtual machines of the source system are indicated to the cloud controller as having higher priority and thus as having priority for a live migration as opposed to a snapshot creation, or any other type of data that may be available to the cloud controller for deciding whether a particular virtual machine 312 should be placed in a first subset or a second subset of virtual machines of the relocation log, 381. Once the relocation log for the virtual machine is generated, the log may be implemented, 384, thereby completing the processing, 386.An embodiment for implementing a relocation protocol for a virtual machine 384 as described herein is described in . Fig. 3C.

[0055] As in Fig. 3C, the implementation of the relocation protocol can enable a live migration of one or more virtual machines (e.g., virtual machine 1 in the example of Fig. 3A) from the source system to the target system, for example, via one or more switches 1 in a data path between the source and target systems, 390. In combination with the live migration of one or more virtual machines, the processing includes a snapshot creation of one or more virtual machines (e.g., virtual machine 2, virtual machine 3) of the source system, 392. In one or more implementations, the snapshot creation process may have a lower priority than the live migration process. In one or more other embodiments, the snapshot creation may occur in parallel with the migration of one or more virtual machines. The snapshot data is forwarded from the source system (for example, via one or more data paths to one or more intermediate systems), which may include one or more hardware components such as switches, 394.After the virtual machine snapshots have been transferred from the intermediate system(s) to the destination system, they can be deployed on the destination system. 396

[0056] Another embodiment of a computing environment that may embody and utilize one or more aspects of the virtual machine relocation facility described herein is described in Fig. 4A. In this example, a computing environment 400 includes, for example, a native central processing unit (CPU) 402, memory 404, and one or more input / output devices and / or interfaces 406 interconnected, for example, via one or more buses 408 and / or other connections. For example, the computing environment 400 may include a PowerPC processor or a Power Systems server offered by International Business Machines Corporation of Armonk, New York; an HP Superdome with Intel Itanium II processors offered by Hewlett Packard Co., Palo Alto, California; and / or other machines based on architectures offered by International Business Machines Corporation, Hewlett Packard, Intel, Oracle, or others.

[0057] Native central processing unit 402 contains one or more native registers 410, such as one or more general-purpose registers and / or one or more special-purpose registers, used in processing within the environment. These registers contain information representing the state of the environment at any given time.

[0058] Furthermore, the native central processing unit 402 executes instructions and code stored in memory 404. In one particular example, the central processing unit executes emulator code 412 stored in memory 404. This code enables the computing environment configured in one architecture to emulate one or more other architectures. For example, the emulator code 412 allows machines based on architectures other than z / Architecture, such as PowerPC processors, Power Systems servers, HP Superdome servers, or others that support z / Architecture ® (and / or ESA / 390) and execute software and instructions based on the z / Architecture ® were developed.

[0059] Further details on the emulator code 412 are provided with reference to Fig. 4B. Guest instructions 450 stored in memory 404 comprise software instructions (e.g., correlating to machine instructions) designed to execute in an architecture other than that of the native CPU 402. For example, the guest instructions 450 may be designed to run on a z / Architecture ®processor, but are instead emulated on the native CPU 402, which may be, for example, an Intel Itanium II processor. In one example, the emulator code 412 includes an instruction fetch routine 452 to obtain one or more guest instructions 450 from memory 404 and optionally provide local buffer storage for the received instructions. It also includes an instruction translation routine 454 to determine the type of guest instruction received and to translate the guest instruction into one or more native instructions 456. This translation includes, for example, identifying the function to be performed by the guest instruction and selecting the native instruction(s) to perform that function.

[0060] Emulator 412 further includes an emulation control routine 460 that causes the native instructions to be executed. Emulation control routine 460 may cause native CPU 402 to execute a routine of native instructions that emulate one or more previously acquired guest instructions and, upon completion of such execution, return control to the instruction fetch routine to emulate the acquisition of the next guest instruction or group of guest instructions. Execution of native instructions 456 may include loading data from memory 404 into a register; storing the data from a register back into memory; or performing some type of arithmetic or logical operation as determined by the translation routine.

[0061] For example, each routine is implemented in software stored in memory and executed by the native central processing unit 402. In further examples, one or more of the routines or operations are implemented in firmware, hardware, software, or a combination thereof. The registers of the emulated processor may be emulated using the registers 410 of the native CPU or using memory locations in memory 404. In embodiments, the guest instructions 450, the native instructions 456, and the emulator code 412 may be located in the same memory or provided on different memory units.

[0062] Exemplary embodiments of further data processing environments for implementing one or more aspects of the present invention are described with reference to Fig. 5 to 7 described

[0063] In another example, Fig. 5 illustrates one embodiment of a computing environment 500 including a computing system 512. Examples of known computing systems, environments, and / or configurations that may be suitable for use with computing system 512 include, but are not limited to, a desktop computer, a workstation, a handheld or laptop computer or device, a mobile phone, a programmable consumer electronics device, a tablet, a PDA, and the like.

[0064] Data processing system 512 may be described in the general context of computer system-executable instructions, such as program modules executed by a computer system. In general, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types.

[0065] As in Fig. 5, the data processing system 512 is shown in the form of a general-purpose data processing unit. The components of the data processing system 512 may include, but are not limited to, one or more processors or processing units 516, a system memory 523, and a bus 518 that connects various system components, including the system memory 523, to the processor 516.

[0066] In one embodiment, processor 516 may be based on the z / Architecture ® offered by International Business Machines Corporation or other architectures offered by International Business Machines Corporation or other companies. z / Architecture ® is a registered trademark of International Business Machines Corporation, Armonk, New York, USA. An embodiment of the z / Architecture ® is described in an IBM publication entitled “z / Architecture ® Principles of Operation," IBM publication SA22-7832-10, March 2015, which is incorporated herein by reference in its entirety.

[0067] In further examples, it may be based on other architectures, such as the Power Architecture offered by International Business Machines Corporation. One embodiment of the Power Architecture is described in "Power ISA™ Version 2.07B," International Business Machines Corporation, April 9, 2015, which is incorporated herein by reference in its entirety. POWER ARCHITECTURE is a registered trademark of International Business Machines Corporation, Armonk, New York, USA. Other names used herein may be registered trademarks, trademarks, or product names of International Business Machines Corporation or other companies.

[0068] Bus 518 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example and not limitation, such architectures include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0069] Data processing system 512 may include a variety of media readable by a computer system. Such media may be any available media accessible by data processing system 512, including both volatile and non-volatile media, removable and non-removable media.

[0070] System memory 523 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 530 and / or cache memory 532. Data processing system 512 may further include other removable / non-removable, volatile / non-volatile computer system storage media. For example only, storage system 534 may be provided for reading from and writing to non-removable, non-volatile magnetic media (not shown and typically referred to as a "hard drive"). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a floppy disk) and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM, or other optical media could be provided.In such cases, all may be connected to bus 518 via one or more data storage interfaces. As described below, memory 523 may contain at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of embodiments of the invention.

[0071] The program / utility 540, comprising a set of (at least one) program modules 542, may be stored, for example and not limited to, in memory 532, as may an operating system, one or more application programs, other program modules, and program data. The operating system, one or more application programs, other program modules, and program data, or a combination thereof, may each include an implementation of a network environment. The program modules 542 generally perform the functions and / or methodologies of embodiments of the invention as described herein. Alternatively, a controller, system, module, logic, etc. 501 for relocating a virtual machine within the computing environment 512 may be separately provided.

[0072] The data processing system 512 may also communicate with one or more external devices 514, such as a keyboard, a pointing device, a display 524, etc.; one or more devices that enable a user to interact with the data processing system 512; and / or any devices (e.g., network card, modem, etc.) that enable the data processing system 512 to communicate with one or more other data processing devices. Such communication may occur via input / output (I / O) interfaces 522. Furthermore, the data processing system 512 may be connected to one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g.,the Internet), via a network adapter 520. As illustrated, the network adapter 520 communicates with the other components of the data processing system 512 via the bus 518. It should be understood that, although not shown, other hardware and / or software components could be used in conjunction with the data processing system 512. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0073] One or more aspects may relate to or use cloud computing.

[0074] It should be understood at the outset that, although this disclosure contains a detailed description of cloud computing, implementation of certain teachings cited herein is not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in connection with any other type of computing environment now known or later developed.

[0075] Cloud computing is a service model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four usage models.

[0076] The features are as follows: On-demand self-service: A cloud user can unilaterally and automatically provision computing resources, such as server time and network storage, as needed, without requiring human interaction with the service provider.

[0077] Broad network access: Functions are available over a network and are accessed through standard mechanisms that support use across heterogeneous thin client or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0078] Resource pooling: The provider's computing resources are pooled to serve multiple users using a multi-tenant model with different physical and virtual resources, which are dynamically allocated or de-allocated as needed. There is a sense of location independence in that the user generally has no control or knowledge regarding the exact location of the deployed resources, but may be able to specify a location at a higher level of abstraction (e.g., country, state, or data center).

[0079] Rapid elasticity: Features can be deployed quickly and flexibly, in some cases automatically, for rapid expansion and quickly released for rapid reduction. To the user, the features available for deployment often appear unlimited and can be purchased in any quantity at any time.

[0080] Properly metered service: Cloud systems automatically monitor and optimize resources by using a metering function at an abstraction level appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and the user of the service.

[0081] There are the following service models: Software as a Service (SaaS): The functionality provided to the user is to use the provider's applications running on a cloud infrastructure. The applications can be accessed from various client devices via a thin client interface, such as a web browser (e.g., web-based email). The user neither manages nor controls the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, except for potentially limited user-specific settings of an application configuration.

[0082] Platform as a Service (PaaS): The functionality provided to the user is to deploy applications created or acquired by the user on the cloud infrastructure using programming languages ​​and tools supported by the provider. The user neither manages nor controls the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and possibly the application hosting environment configurations.

[0083] Infrastructure as a Service (IaaS): The functionality provided to the user consists of providing processing, storage, networking, and other basic computing resources, allowing the user to deploy and run any software, including operating systems and applications. The user neither manages nor controls the underlying cloud infrastructure, but has control over operating systems, storage, and the applications used, and possibly limited control over selected network components (e.g., host firewalls).

[0084] There are the following usage models: Private Cloud: The cloud infrastructure is operated exclusively for an organization. It can be managed by the organization or a third party and can be located on or off-premises.

[0085] Community Cloud: The cloud infrastructure is shared by multiple organizations and supports a specific community that shares common problems (e.g., considering objectives, security requirements, policies, and compliance). It can be managed by the organizations or a third party and can be located on-premises or off-premises.

[0086] Public Cloud: The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization that sells cloud services.

[0087] Hybrid Cloud: The cloud infrastructure is a composition of two or more clouds (private, community or public) that remain distinct entities but are united by a standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load balancing between clouds).

[0088] A cloud computing environment is service-oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure comprising a network of interconnected nodes.

[0089] A cloud computing node may include a computer system / server, such as in Fig. 5. The computer system / server 512 of Fig. 5 may operate in distributed cloud computing environments, with tasks being executed by remotely located processing units connected via a data transmission network. In a distributed cloud computing environment, program modules may be located on both local and remotely located computer system storage media containing storage units. The computer system / server 512 may implement and / or perform any of the functionalities discussed above.

[0090] With reference to Fig. 6 illustrates a cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by users of the cloud, such as personal digital assistant (PDA) or mobile phone 54A, desktop computer 54B, laptop computer 54C, and / or vehicle computer system 54N, can exchange data. The nodes 10 can exchange data with each other. They can be physically or virtually grouped in one or more networks (not shown), such as private, community, public, or hybrid clouds, as described hereinabove, or a combination thereof. Thus, the cloud computing environment 50 has the ability to offer infrastructure, platforms, and / or software as services for which a cloud user does not need to maintain any resources on a local computing device. It should be understood that the Fig. 6 are intended to be illustrative only, and that the computing nodes 10 and the cloud computing environment 50 may communicate with any type of computerized device over any type of network and / or a network-addressable connection (e.g., using a web browser).

[0091] With reference to Fig. 7 shows a group of functional abstraction layers provided by the cloud computing environment 50. It should be clear from the outset that the Fig. The components, layers, and functions shown in Figure 7 are intended to be illustrative only, and embodiments of the invention are not limited thereto. As shown, the following layers and corresponding functions are provided: The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframes 61; servers based on a RISC (Reduced Instruction Set Computer) architecture 62; servers 63; blade servers 64; storage devices 65; and networks and networked components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0092] A virtualization layer 70 provides an abstraction layer from which the following example virtual entities may be deployed: virtual servers 71; virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0093] In one example, the management layer 80 may provide the functions described below. Resource provisioning 81 provides for dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 82 enable cost tracking and billing or invoicing for the use of resources within the cloud computing environment. In one example, these resources may include licenses for application software. Security provides identity verification for users of the cloud and tasks, as well as protection for data and other resources. A user portal 83 provides access to the cloud computing environment for users and system administrators.Service Level Management (84) ensures the allocation and management of cloud computing resources so that required service levels are met. Service Level Agreement (SLA) planning and contract fulfillment (85) provides advance agreement for and procurement of cloud computing resources for which future demand is expected according to an SLA.

[0094] The workload layer 90 provides examples of the functionality for which the cloud computing environment can be used. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual training delivery 93; data analytics processing 94; transaction processing 95; and a processing unit (PU) for handling the relocation of virtual machines 96.

[0095] The descriptions of the various embodiments of the present invention have been prepared for the purpose of illustration, but are by no means intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, practical application, or technical improvement over current technologies, or to enable others skilled in the art to understand the disclosed embodiments.

[0096] The present invention may be a system, a method, and / or a computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or storage media) having computer-readable program code thereon for causing a processor to carry out aspects of the present invention.

[0097] The computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution unit. A computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.A non-exhaustive list of more specific examples of the computer-readable storage medium includes the following: a portable computer diskette, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), a portable CD-ROM, a DVD drive (DVD), a memory stick, a floppy disk, a mechanically encrypted device such as punched cards or raised structures in a groove with instructions recorded thereon, and any suitable combination of the foregoing. A computer-readable storage medium, as used herein, need not be designed to carry transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g.,light pulses passing through a fiber optic cable) or electrical signals transmitted through a wire.

[0098] Computer-readable program instructions described herein may be downloaded to respective computing / processing units from a computer-readable storage medium or to an external computer or storage device via a network, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission lines, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing unit receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing unit.

[0099] Computer-readable program instructions for performing operations of the present invention may be assembly language instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or other source code or object code written in any combination of one or more programming languages, including Smalltalk, C++, or the like, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remotely located computer, or entirely on the remotely located computer or server.In the latter scenario, the remote computer may be connected to the user's computer over any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., over the Internet using an Internet service provider). In some embodiments, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute the computer-readable program instructions using state information of the computer-readable program instructions to personalize the electronic circuit to perform aspects of the present invention.

[0100] Aspects of the present invention are described herein with reference to flowchart and / or block diagram illustrations of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block in the flowchart and / or block diagram illustrations, and combinations of blocks in the flowchart and / or block diagram illustrations, may be implemented by computer-readable program instructions.

[0101] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other devices that process programmable data to produce a machine such that the instructions, executing via the processor of the computer or other devices that process programmable data, create means for implementing the functions / acts specified in the flowchart and / or the block or blocks of the block diagram.These computer-readable program instructions may also be stored in a computer-readable storage medium that can control a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable storage medium having the instructions stored therein comprises an article of manufacture, including instructions that implement the function / act specified in the flowchart and / or the block or blocks of the block diagram.

[0102] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operations to be performed on the computer, other programmable device, or other device to produce a computer-implemented process, such that the instructions executing on the computer, other programmable device, or other device implement the functions / acts specified in the flowchart and / or the block or blocks of the block diagram.

[0103] The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or section of instructions comprising one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions specified in the blocks may occur out of the order specified in the figures. For example, two blocks shown consecutively may actually execute substantially in parallel, or the blocks may sometimes execute in the reverse order, depending on the functionality involved.It is also noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or by combinations of special purpose hardware and computer instructions.

[0104] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It is further intended to be clear that the terms "comprise" (and any form of comprise such as "comprises" and "comprising"), "have" (and any form of have such as "has" and "having"), "contain" (and any form of contain such as "contains" and "containing"), and "include" (and any form of include such as "includes" and "including") are open linking verbs. As a result, a method or device that "comprises," "has," "contains," or "includes" one or more steps or elements has, but is not limited to, only one or more steps or elements.to have this one or more steps or elements. Similarly, a step of a method or an element of an entity that "exhibits," "has," "contains," or "includes" one or more features has this one or more features, but is not limited to having only this one of the multiple features. Furthermore, an entity or structure configured in a particular way is configured at least in that way, but may also be configured in other, unlisted ways.

[0105] The corresponding structures, materials, acts, and equivalents of all means or step-plus-function elements in the following claims are intended to include, if any, all structures, materials, or acts for performing the function in combination with other claimed elements, as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is by no means intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention.The embodiment was chosen and described in order to best explain the principles of one or more aspects of the invention and the practical application, and to enable others skilled in the art to understand one or more aspects of the invention for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

[1] A method implemented by a computer, comprising: Obtaining (275), by a first controller (211), first power-related information from a first power system group (201) comprising one or more first power systems that provide power to a first group (200) of powered hardware components associated with the first controller, the first group of powered hardware components executing a first plurality of virtual machines (212); and Generating (280), by the first controller (211), a relocation protocol to migrate the first plurality of virtual machines (212) based at least in part on the first power-related information, the relocation protocol including: a migration (281) of a first subset of one or more virtual machines of the first plurality of virtual machines (212) such that the first subset of one or more virtual machines is migrated to a second group (220) of powered hardware components and is performed such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration; and creating a snapshot (282) of a second subset of one or more virtual machines of the first plurality of virtual machines (212). [2] A computer-implemented method according to claim 1, further comprising: Migrating (285) the first subset of one or more virtual machines to the second group (220) of powered hardware components in a manner such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration; and Migrating (290) the second subset of one or more virtual machines by at least partially providing the second subset of one or more virtual machines (232) to the second group (220) of powered hardware components based on one or more snapshots of the second subset of one or more virtual machines obtained at the first group (200) of powered hardware components. [3] The computer-implemented method of claim 2, wherein migrating (290) the second subset of one or more virtual machines is performed after completing migrating the first subset of one or more virtual machines. [4] The computer-implemented method of claim 2, wherein migrating (290) the second subset of one or more virtual machines includes first transferring the respective snapshot(s) of the virtual machine(s) of the second subset of one or more virtual machines to an intermediate group of powered hardware components powered by an intermediate power system group (350) before provisioning the second subset of one or more virtual machines to the second group (220) of powered hardware components based on their respective snapshot(s), wherein the intermediate power system group (350) is different from the first power system group (201). [5] The computer-implemented method of claim 4, wherein transferring (394) the respective snapshot(s) of the virtual machine(s) of the second subset of one or more virtual machines to the intermediate group (350) of powered hardware components occurs substantially simultaneously with migrating (285) the first subset of one or more virtual machines to the second group (220) of powered hardware components. [6] The computer-implemented method of claim 2, wherein the first group (200) of powered hardware components belongs to a first cloud (210), the first controller being a first cloud controller (211), and the second group (220) of powered hardware components belongs to a second cloud (230), the second group of powered hardware components being powered by a second power system group (221) comprising one or more second power systems. [7] The computer-implemented method of claim 6, wherein the first cloud (210) and the second cloud (230) are hosted in different, geographically separate data centers. [8] The computer-implemented method of claim 6, wherein the second cloud (230) includes a second cloud controller (231), and generating (280) further comprises obtaining, by the first cloud controller (211), second power-related information from the second cloud controller (231) or the second power system group (221) indicative of the power state for the second group (220) of powered hardware components. [9] The computer-implemented method of claim 1, wherein generating (280) the relocation log by the first controller (211) includes referencing (382) history data related to at least one of a migrating virtual machine or a snapshot creation of a virtual machine to enable placing at least one of the first plurality of virtual machines (212) in either the first subset of one or more virtual machines or the second subset of one or more virtual machines. [10] The computer-implemented method of claim 1, wherein generating (280) the relocation log by the first controller (211) includes using associated priority information of at least one virtual machine of the first plurality of virtual machines (212) for placing the at least one virtual machine in either the first subset of one or more virtual machines or the second subset of one or more virtual machines. [11] The computer-implemented method of claim 1, wherein generating (280) the relocation log by the first controller (211) includes determining (374) the power state for one or more switches (322, 326) in a data transmission path from the first group (200) of powered hardware components to the second group (220) of powered hardware components. [12] The computer-implemented method of claim 1, wherein the first power-related information includes an estimated amount of time until power failure for the first group (200) of powered hardware components, and the relocation log is an emergency relocation log dynamically generated (280) based at least in part on the amount of time until power failure of the first group (200) of powered hardware components. [13] A system (512) for enabling relocation of virtual machines (212), the system comprising: a working memory (523); and a processing circuit (516) connected in data exchange with the main memory (523), wherein the system (512) executes a method comprising: Obtaining (275), by a first controller (211), first power-related information from a first power system group (201) comprising one or more first power systems that provide power to a first group (200) of powered hardware components associated with the first controller (211), the first group (200) of powered hardware components executing a first plurality of virtual machines (212); and Generating (280), by the first controller (211), a relocation protocol to migrate the first plurality of virtual machines (212) based at least in part on the first power-related information, the relocation protocol including: a migration (281) of a first subset of one or more virtual machines of the first plurality of virtual machines (212) such that the first subset of one or more virtual machines is migrated to a second group (220) of powered hardware components and is performed such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration; and creating a snapshot (282) of a second subset of one or more virtual machines of the first plurality of virtual machines (212). [14] The system (512) of claim 13, wherein the first power-related information includes an estimated amount of time until a power failure for the first group (200) of powered hardware components, and the relocation log is an emergency relocation log that is dynamically generated (280) based at least in part on the amount of time until a power failure of the first group (200) of powered hardware components. [15] A computer program product for enabling relocation of virtual machines (212), the computer program product comprising: a computer-readable storage medium readable by a processing circuit (516) and storing instructions for execution by the processing circuit to perform a method comprising: Obtaining (275), by a first controller (211), first power-related information from a first power system group (201) comprising one or more first power systems that provide power to a first group (200) of powered hardware components associated with the first controller (211), the first group (200) of powered hardware components executing a first plurality of virtual machines (212); and Generating (280), by the first controller (211), a relocation protocol to migrate the first plurality of virtual machines (212) based at least in part on the first power-related information, the relocation protocol including: a migration (281) of a first subset of one or more virtual machines of the first plurality of virtual machines (212) such that the first subset of one or more virtual machines is migrated to a second group (220) of powered hardware components and is performed such that the first subset of one or more virtual machines continues to operate in a substantially continuous manner during the migration; and creating a snapshot (281) of a second subset of one or more virtual machines of the first plurality of virtual machines (212).

Images