Method, device and computer program product for migrating virtual machine

Abstract

The embodiment of the invention relates to a method for migrating a virtual machine, electronic equipment and a computer program product. The method includes obtaining a source operating state of the virtual machine on a data repository of a source type. The method also includes determining a plurality of candidate migration actions to migrate the virtual machine from the source type of data repository to the plurality of types of data repository, respectively. The method also includes determining a plurality of action scores based on the source operating state and the plurality of candidate migration actions. The method further includes selecting a target action based on the plurality of action scores. The method further includes migrating the virtual machine from the data repository of the source type to the data repository of the target type indicated by the target action by performing the target action. In this way, the optimal type of the data repository for the virtual machine is found out by considering the operating state of the virtual machine, so that the performance of the virtual machine is enhanced, and better virtualization service is achieved.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to the field of data storage, and more specifically to methods, apparatus, and computer program products for migrating virtual machines. Background Technology

[0002] A virtual machine (VM) is a virtual environment created on top of a physical hardware system using virtualization technology. It acts as a virtual computer system, simulating the entire hardware of a computer, including the CPU, memory, network interface, and storage. Through appropriate management software, resources and hardware can be separated and properly configured for the VM's use.

[0003] Virtualization technology allows multiple virtual environments to share a single system. A hypervisor is configured to manage hardware and separate physical resources from the virtual environments. Resources from the physical environment are partitioned as needed and then allocated to the virtual machines. For example, physical storage resources are mapped to logical storage resource units, i.e., datastores, and then the hypervisor allocates the datastores to the virtual machines. Summary of the Invention

[0004] Embodiments of this disclosure provide methods, apparatus, and computer program products for migrating virtual machines. In a first aspect of this disclosure, a method for migrating a virtual machine is provided. The method includes obtaining a source operational state of the virtual machine on a data repository of a source type. The method further includes determining a plurality of candidate migration actions for migrating the virtual machine from the source type data repository to multiple types of data repositories, respectively. The method further includes determining a plurality of action scores for the plurality of candidate migration actions based on the source operational state and the plurality of candidate migration actions, the action scores indicating the operational performance of the virtual machine on the migrated data repository. The method further includes selecting a target action from the plurality of candidate migration actions based on the plurality of action scores. The method further includes migrating the virtual machine from the source type data repository to a data repository of a target type indicated by the target action by executing the target action.

[0005] In a second aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the electronic device to perform an action, the action including acquiring a source operation state of a virtual machine on a data repository of a source type. The action also includes determining a plurality of candidate migration actions for migrating the virtual machine from the data repository of the source type to multiple data repositories of various types. The action further includes determining a plurality of action scores for the plurality of candidate migration actions based on the source operation state and the plurality of candidate migration actions. Here, the action score indicates the operational performance of the virtual machine on the migrated data repository. The action also includes selecting a target action from the plurality of candidate migration actions based on the plurality of action scores. The action further includes migrating the virtual machine from the data repository of the source type to a data repository of a target type indicated by the target action by executing the target action.

[0006] In a third aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions. When executed by a device, the machine-executable instructions cause the device to perform any step of the method described in the first aspect of this disclosure.

[0007] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Attached Figure Description

[0008] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0009] Figure 1 A schematic diagram of an example system that can be implemented therein according to some embodiments of the present disclosure is shown;

[0010] Figure 2 A flowchart illustrating an example method for migrating virtual machines according to some embodiments of this disclosure is shown;

[0011] Figure 3 A schematic diagram of an example apparatus for determining migration actions according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A flowchart illustrating an example method for training a transfer model according to some embodiments of this disclosure is shown; and

[0013] Figure 5 A schematic block diagram of an example device that can be used to implement embodiments of the present disclosure is shown.

[0014] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0015] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0016] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0017] As discussed above, data repositories can be used to store virtual machine operating system files, application files, data files, etc. In other words, virtual machines are deployed in data repositories. A virtual system can include various types of data repositories.

[0018] Because different types of data repositories use different protocols, virtual machines (VMs) achieve varying performance levels when running on them. This performance difference can depend on many factors, including workload, storage hardware, and configuration. Furthermore, the impact of snapshots on VM performance is increasingly being recognized. Therefore, factors influencing VM performance can generally include, for example, workload, data repository type, number of snapshots, and hardware configuration.

[0019] In some read-intensive workloads, initially, a virtual machine might perform better running on one type of data repository than on another. However, as the number of snapshots increases, its performance will significantly decrease and fall below that of the other type of data repository. At this point, the virtual machine management module can recommend migrating the virtual machine. In related technologies, virtual machine load balancing mechanisms determine the target migration location for virtual machines based on CPU, network, storage, and other resource usage. However, this does not take into account the impact of workload, data repository type, number of snapshots, and hardware configuration.

[0020] However, considering all the above factors, choosing a suitable data repository becomes extremely difficult and complex, because not only do workload types vary, but software / hardware configurations are also complex, and their impact is hard to define. At this point, setting the optimal data repository for different applications with diverse workloads, hardware, and configurations (such as the number of snapshots) relies heavily on the experience and subjective judgment of operations personnel, rather than objectively considering all relevant factors.

[0021] In view of this, embodiments of this disclosure propose a scheme for selecting a target type data repository based on action scores to address one or more of the aforementioned problems and other potential issues. In this scheme, candidate migration actions for migrating to an available data repository can be determined based on the current source type data repository. Then, based on the virtual machine's operational status on the source type data repository, an action score for executing each candidate migration action can be determined. The action score can indicate the performance of the virtual machine running on the new data repository after executing the migration action, thereby allowing the selection of performance-enhancing candidate migration actions and their execution.

[0022] Furthermore, in this scheme, the action scoring table is dynamically updated as the number of migrations accumulates, and the empirical data in the action scoring table further guides virtual machine migrations. In the long run, the highest virtual machine performance can be achieved with fewer migrations. In this way, by considering the operating state of the virtual machine, the optimal type of data repository for the virtual machine is identified, thereby enhancing virtual machine performance and achieving better virtualization services. Moreover, compared with methods in related technologies, the embodiments of this disclosure comprehensively consider the operating state of the virtual machine to cover all possible influencing factors to achieve optimal virtual machine performance. Furthermore, the solutions of the embodiments of this disclosure do not rely on the experience of operations and maintenance personnel, improving the accuracy of selection.

[0023] The following will combine Figures 1 to 5 This document describes in detail the basic principles and several example embodiments of the virtual machine migration scheme according to this disclosure. Figure 1A schematic diagram of an example system 100 that can be implemented according to some embodiments of the present disclosure is shown. System 100 includes three types of data repositories, including a first type data repository 102, a second type data repository 104, and a third type data repository 106. In some embodiments, the first type data repository 104 may be, for example, a virtual volume (vVol). vVol is a framework for the virtualization integration and management of Storage Area Network (SAN) and Network Attached Storage (NAS). This framework provides a more efficient model for managing virtualized environments, while transforming the data center from an infrastructure-centric to an application-centric approach. This allows virtualization technology to better serve applications.

[0024] The second type of data repository 102 could be, for example, a Virtual Machine File System (VMFS). VMFS is a high-performance, clustered file system that provides storage virtualization optimized for virtual machines. Each virtual machine is encapsulated in a small set of files; VMFS is the default storage management interface for these files on physical disks and partitions. By efficiently storing the entire machine state in a central location, VMFS enables IT organizations to significantly simplify virtual machine configuration. VMFS reduces management overhead by providing an efficient virtualization management layer particularly suitable for large enterprise data centers.

[0025] The third type of data repository 106 could be, for example, a Virtual Storage Area Network (vSAN). vSAN is a distributed software layer that runs natively. vSAN aggregates local or directly connected capacity devices in a host cluster and creates a single storage pool shared among all hosts in the vSAN cluster. vSAN uses a software-defined approach to create shared storage for virtual machines. The host's local physical storage resources can be virtualized and transformed into storage pools, which can then be partitioned and allocated to the virtual machines and applications based on their quality of service requirements.

[0026] Virtual machine VM 108-1 runs on data repository 102 of type 1. VM 108-2 runs on data repository 104 of type 2. VM 108-3 runs on data repository 106 of type 3.

[0027] As discussed above, due to the different types of data repositories, the overall performance will vary during operation. Figure 1In the illustrated embodiment, Figure 110 shows the performance variation of the same application running on three types of data repositories within a virtual machine. The curves in Figure 110 represent the variation in input / output per second (IOPS) with the number of snapshots. Curve 112 shows the performance variation of the virtual machine running on the first type of data repository 102, curve 114 shows the performance variation of the virtual machine running on the second type of data repository 104, and curve 116 shows the performance variation of the virtual machine running on the third type of data repository 106.

[0028] As can be seen, at the beginning of the operation, when the number of snapshots is relatively small (T1), the virtual machine's performance on the second type of data repository 104 is significantly higher than its performance on the other two types of data repositories. However, as the number of snapshots increases, the virtual machine's performance on the second type of data repository 104 continuously decreases. At point T2, when the number of snapshots reaches 5, the virtual machine's performance on the second type of data repository 104 is equal to its performance on the first type of data repository 102, and it is foreseeable that the performance of the virtual machine on the second type of data repository 104 will continue to decline and become lower than its performance on the first type of data repository 102. At this point, virtual machine 108-2 can issue a migration request to the virtual machine management module to trigger a migration action.

[0029] Upon receiving the migration request from virtual machine 108-2, the virtual machine management module determines that system 100 includes three types of data repositories, allowing virtual machine 108-2 to be migrated to a first-type data repository 102 by executing a first migration action A1. Virtual machine 108-2 can also be retained in a second-type data repository 104 by executing a second migration action A2. Virtual machine 108-2 can be migrated to a third-type data repository 106 by executing a third migration action A3. The virtual machine management module determines the operation state S2(T2) of virtual machine 108-2 at time T2 operating in the second-type data repository 104. Based on the operation state S2(T2), the virtual machine management module can find the action score Q1(S2,A1) for executing the first migration action A1, the action score Q1(S2,A2) for executing the second migration action A2, and the action score Q3(S2,A3) for executing the third migration action A3 in the action score table 120.

[0030] Here, the action score can indicate the performance metrics of the virtual machine running on the migrated data repository. For example, at time T3 after executing the first migration action A1, virtual machine 108-2 is migrated to data repository 102 of type 1. At this time, according to Figure 110, virtual machine 108-2 has the highest running performance on data repository 102 of type 1. Therefore, the action score Q1(S2,A1) has the highest value. At time T3 after executing the second migration action A2, virtual machine 108-2 is retained in data repository 104 of type 2. At this time, according to Figure 110, virtual machine 108-2 has the lowest running performance on data repository 104 of type 2. Therefore, the action score Q2(S2,A2) has the lowest value. At time T3 after executing the third migration action A3, virtual machine 108-2 is migrated to data repository 106 of type 3. At this time, according to Figure 110, virtual machine 108-2 has the second highest running performance on data repository 102 of type 3. Therefore, the action score Q1(S2,A1) has the second highest value.

[0031] Therefore, the virtual machine management module can determine the highest benefit from executing the first migration action A1 based on the action score, and thus select and execute the first migration action A1. In this way, for example, when a virtual machine autonomously sends a migration request to trigger a data repository migration, the system 100 can determine the optimal target type of data repository and migration action based on the action score, thereby automating the virtual machine migration.

[0032] It should be understood that Figure 1 The system 100 shown is merely exemplary and not limiting. The storage system according to this disclosure may also have other forms or structures.

[0033] The basic principles and several exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. Figure 2 A flowchart of an example method 200 for migrating virtual machines according to some embodiments of the present disclosure is shown. For ease of explanation, reference will be made to... Figure 1 Method 200 is described herein. Method 200 may be implemented by system 100 or a virtual machine management module within the system. It should be understood that method 200 may also be performed by other suitable devices or apparatuses. Method 200 may include additional actions not shown and / or the actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0034] like Figure 2 As shown, at 202, method 200 includes obtaining the source operation state of the virtual machine on the data repository of the source type. For example, in Figure 1In the illustrated embodiment, system 100 can obtain the source operation state S2(T2) of virtual machine 108-2 on data repository 104 of source class 2. The operation state includes parameters that reflect the operating performance of the virtual machine, such as latency or throughput. The operation state may also include environmental parameters related to the operating performance of the virtual machine, such as hardware configuration.

[0035] At 204, method 200 includes identifying multiple candidate migration actions for migrating virtual machines from a source type data repository to multiple types of data repositories, respectively. For example, in Figure 1 In the illustrated embodiment, system 100 may determine that system 100 includes three types of data repositories, and determine that candidate migration actions for the current virtual machine include a first migration action A1, a second migration action A2, and a third migration action A3.

[0036] At point 206, method 200 includes determining multiple action scores for multiple candidate migration actions based on the source operation state and multiple candidate migration actions. Here, the action scores indicate the operational performance of the virtual machine on the migrated data repository. For example, in Figure 1 In the illustrated embodiment, system 100 can determine three action scores Q1, Q2, and Q3 based on the source operation state S2(T2) and the first migration action A1, the second migration action A2, and the third migration action A3, respectively. In some embodiments, the action score table can be generated by directly recording the system's historical operation information. In some alternative embodiments, the action score table can also be an action score function obtained from the system's historical operation information according to a specific fitting method. The action score function is, for example, a function for the operation state and the migration action.

[0037] At point 208, method 200 includes selecting the target action from multiple candidate transfer actions based on multiple action scores. For example, in Figure 1 In the illustrated embodiment, system 100 can select the migration action A1 with the highest action score Q1. In some embodiments, the system can also select migration actions with action scores greater than a predetermined score threshold.

[0038] At 210, method 200 includes migrating a virtual machine from a data repository of the source type to a data repository of the target type indicated by the target action by performing a target action. For example, in Figure 1 In the illustrated embodiment, system 100 can perform migration action A1 to migrate virtual machine 108-2 from source type data repository 104 to target type data repository 102.

[0039] exist Figure 2In the illustrated embodiments, the optimal type of data repository for the virtual machine is determined by considering its operational state, thereby enhancing virtual machine performance and achieving better virtualization services. Furthermore, the solutions of this disclosure can be performed autonomously without relying on the experience of operations personnel, ensuring the timeliness of virtual machine migration and improving the accuracy of selecting the target type of data repository.

[0040] In some embodiments, the system can also recommend optional data repositories with action scores within a certain threshold range to the user, without directly performing migration actions. This allows users to promptly select their preferred data repository, thereby improving the user experience.

[0041] As discussed above, in related technologies, for different applications (with different workloads) on different virtual machines, users typically need experienced operators to try several steps to find the optimal data storage. Furthermore, even experienced operators may not be able to identify a suitable migration target. Therefore, embodiments of this disclosure also propose a reinforcement learning-based framework for efficiently obtaining an action evaluation table. In this reinforcement learning-based framework, it is assumed that the virtual machine runs on a data repository of a certain type. Here, the goal is to find an optimal data repository for the virtual machine to maximize its performance, or to achieve the target performance through multiple migrations of the virtual machine.

[0042] In a reinforcement learning-based framework, an agent is configured. The agent observes the virtual machine's operational state s(t) at time step t. Then, the agent selects a transition action A(t) according to an action selection policy and transitions to the next state s(t+1) at the next time step t+1. The agent calculates the reward r(t+1) based on state s(t+1). Therefore, the interval between two states is one time step. The agent then uses algorithms including, but not limited to, Q-learning, Deep Q-Network (DQN), and Dual DQN (DDQN) to update the action evaluation values ​​or action evaluation functions Q(s,a) in the action evaluation table. The action evaluation function or action evaluation value defines the long-term value of taking action a in any state s. Over time, the agent learns to pursue actions that maximize cumulative reward or bonus in any state.

[0043] The following will combine Figures 3 to 5 This describes a reinforcement learning-based framework for efficiently acquiring action evaluation tables. Figure 3 A schematic diagram of an example apparatus 300 for determining migration actions according to some embodiments of the present disclosure is shown. Figure 3As shown, the apparatus 300 includes a data repository migration agent 302. The data repository migration agent 302 is a reinforcement learning-based agent that autonomously selects data repositories, aiming to find the optimal data repository to maximize virtual machine performance.

[0044] The data repository migration agent 302 includes a data repository selection unit 304. The data repository selection unit 304 observes the current operating state s(t) of the virtual machine 312 and provides migration operations for the data repository. The data repository migration agent 302 also includes an action scoring table module 306. The action scoring table module 306 uses a reinforcement learning algorithm to update the action score based on the current state of the virtual machine 312, the migration action for the virtual machine 312, the reward after migrating the virtual machine 312, and the state of the virtual machine 312 at the next time step. The data repository migration agent 302 also includes a state detection module 308. The state detection module 308 acquires the operating state of the virtual machine 312. The operating state includes, for example, static and real-time information such as virtual machine information, hardware configuration, runtime performance matrices (e.g., IO latency / IOPS / CPU utilization), workload (e.g., IO size / read / write ratio), number of snapshots, and current data storage. The data repository migration agent 302 also includes a reward calculation module 310. The reward calculation module 310 calculates the action reward in state s(t) according to the objective: to find the optimal data repository that maximizes the performance of the virtual machine.

[0045] This concludes the description of an example device 300 for implementing a reinforcement learning-based framework for efficiently acquiring an action evaluation table. Further details will follow. Figure 4 This describes a method for efficiently obtaining action evaluation tables. Figure 4 A flowchart illustrating an example method for training a transfer model according to some embodiments of this disclosure is shown. For example, method 400 may be provided by... Figure 3 The method is implemented by device 300. It should be understood that method 400 can also be performed by other suitable devices or apparatus, such as... Figure 1 The method 400 may include additional actions not shown and / or may omit the actions shown, and the scope of this disclosure is not limited in this respect.

[0046] like Figure 4As shown, at position 402, device 300 customizes its training strategy. The training strategy includes: an acceptable list of data repositories, a maximum number of training attempts, and an acceptable performance range, such as latency less than 5ms. At position 404, device 300 retrieves a migration request from the virtual machine to migrate the virtual machine. This request may be automatically triggered by a virtual machine experiencing performance issues, indicating that migration should occur. For example, for an application on VMFS with a workload of 512KB sequential read IOPS, a migration request will be generated due to performance degradation if the number of snapshots exceeds 5.

[0047] At 406, device 300 detects a first operational state of the virtual machine in a first type of data repository. Here, the operational state is a vector at a given time step. It represents the static and real-time system states of the virtual machine at time step t. In some embodiments, the operational state may include states represented as:

[0048] s = {VM info ,Snum,DS,WL,HD conf.,rt info} (1)

[0049] VM info This indicates virtual machine information; Snum indicates the number of snapshots; DS indicates an optional data repository type; WL indicates the workload; HD conf indicates the hardware configuration; rt info This indicates runtime information.

[0050] In the illustrated embodiment, virtual machine information is a static value representing information about the virtual machine, including, but not limited to, operating system type, number of CPUs, memory size, and hard disk size. Data storage determines the application format, such as VMFS, vSAN, and vVol. Workload represents average I / O information over a given time period, including, but not limited to, I / O size, read / write ratio, and I / O type (e.g., random or sequential). Hardware configuration represents the storage system hardware configuration, such as hardware, platform, and driver information. Runtime information represents the average runtime state (e.g., performance state) over a time step t, such as average total throughput, average CPU utilization, and a rounded value for average latency. An example state is listed below:

[0051] <Virtual Machine Information>

[0052] Operating system = CentOS 7 (64-bit)

[0053] CPU = 4

[0054] Memory = 4GB

[0055] Hard drive = 300GB

[0056] <Data Storage> VMFS

[0057] <Workload>

[0058] Average IO size = 512 (KB)

[0059] Read / write ratio = 95 (read percentage)

[0060] Random / Order Ratio = 0 (Percentage of Randomness)

[0061] <Snapshot_number> 6

[0062] <Hardware Configuration>

[0063] System Hardware: <Hardware>

[0064] System Platform: <Platform>

[0065] Drive information: <drive>

[0066] <Runtime Information>

[0067] Average total IOPS = 50 (K)

[0068] Average throughput = 250 (MB / s)

[0069] Average CPU usage = 70%

[0070] Average delay = 2 (ms).

[0071] At 408, device 300 determines whether the first operation state is in the action scoring table. If device 300 determines that the first operation state is in the action scoring table, then method 400 proceeds to 410. At 410, device 300 obtains the probability of performing a random selection. If device 300 determines that the first operation state is not in the action scoring table, then method 400 proceeds to 412. At 412, the probability of performing a random selection is set to 1. Here, device 300 can choose to perform a random selection or a directed selection based on the probability. Random selection corresponds to "exploration," while directed selection corresponds to "utilization." When an operation state is not recorded in the action scoring table, that is, when no action score indicates the merit of the transition action for the current operation state, device 300 has no information to "utilize" and can only choose "exploration." Device 300 sets the probability to 1, making the performance of a random selection inevitable.

[0072] Conversely, when an operational state is recorded in an action scoring table, meaning the action scoring table includes action scores that indicate the merits of transition actions for the current operational state, the device 300 possesses information that can be "utilized." However, if the device 300 always selects information that has already been recorded, it will be impossible to iterate through all choices. Therefore, the device 300 still needs to explore. In this case, the probability of performing a random selection is set in the range of 0 to 1.

[0073] At 414, device 300 determines, based on probability, whether to perform a random selection or a directed selection. If device 300 determines, based on probability, to perform a directed selection, then method 400 proceeds to 416. At 416, device 300 selects the migration action with the highest action score. If device 300 determines, based on probability, to perform a random selection, then method 400 proceeds to 418. At 418, device 300 randomly selects a migration action from a plurality of feasible migration actions. In some embodiments, feasible migration actions can be migration actions in an action space. For example, in a system that includes three types of data repositories—VMFS, vSAN, and vVol—the action space can be:

[0074] Table 1 Action Space

[0075]

[0076]

[0077] For example, if the current data repository is VMFS, then the possible actions in the action space are: VMFS->vSAN, VMFS->vVol, VMFS->VMFS.

[0078] After a migration action is selected, at 420, device 300 executes an operation to migrate the virtual machine from a first type of data repository to a second type of data repository. At 422, device 300 detects a second operational state of the virtual machine on the second type of data repository. At 424, device 300 calculates a reward based on the first operational state, the second operational state, and the selected migration action.

[0079] In some embodiments, a reward function R can be defined to guide the agent to find a good solution for a given objective. Since the objective is to find the optimal data storage that maximizes the performance of the virtual machine, the reward R can be defined as follows:

[0080]

[0081] Where W1 and W2 are the corresponding weights; L AThis represents the average latency, which is the average latency during the time it takes for a virtual machine to migrate from a data repository of type 1 to a data repository of type 2 (corresponding to the time between detecting the first operational state and detecting the second operational state); L I This represents the initial latency, i.e., the latency of the virtual machine in the first type of data repository; T A T represents the average throughput, specifically the average throughput during the period when virtual machines migrate from a first-type data repository to a second-type data repository; I This represents the initial throughput, i.e., the throughput of the virtual machine in the first type of data repository. The values ​​of W1 and W2 depend on the user's level of concern regarding latency and throughput.

[0082] At point 426, device 300 updates the action score based on a first operating state, a second operating state, the selected transfer action, and the reward, according to a reinforcement learning formula. In some embodiments, the formula used for updating may be an update formula from a Q-learning algorithm, a DQN learning algorithm, or a DDQN learning algorithm. As training progresses, the action score will tend to stabilize and thus converge.

[0083] At point 428, device 300 determines whether the target has been achieved. For example, whether the operating performance is within a predefined acceptable range. If device 300 determines that the target has not been achieved, then method 400 proceeds to 430. At 430, device 300 determines whether the maximum number of common sense attempts has been reached, for example, 3. If device 300 determines that the maximum number of common sense attempts has not been reached, then method 400 proceeds to 432. At 432, device 300 sets the current second operating state to the first operating state to return to point 408, thereby triggering a new iteration.

[0084] If device 300 determines that the goal has been reached, then method 400 proceeds to 434. Furthermore, if device 300 determines that the maximum number of common sense attempts has been reached, then method 400 also proceeds to 434. At 434, device 300 reduces the probability of performing a random selection and ends the training round.

[0085] Figure 5 A schematic block diagram of an example device 500 that can be used to implement embodiments of the present disclosure is shown. For example, such as Figure 1 The storage system 100 shown can be implemented by device 500. For example... Figure 5As shown, device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 502 or loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. CPU 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0086] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] The various processes and handling described above, such as methods 200 and 400, can be executed by processing unit 501. For example, in some embodiments, methods 200 and 400 can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by CPU 501, one or more actions of method 300 described above can be performed.

[0088] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0089] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0090] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0091] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0092] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0094] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0096] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for migrating virtual machines, comprising: Retrieve the source operation status of the virtual machine on the data repository of the source type; Identify multiple candidate migration actions to migrate the virtual machine from the source type of data repository to multiple types of data repositories; Based on the source operation state and the multiple candidate migration actions, multiple action scores are determined for the multiple candidate migration actions. The action scores indicate the operational performance of the virtual machine on the migrated data repository. Based on the multiple action scores, a target action is selected from the multiple candidate migration actions; as well as By executing the target action, the virtual machine is migrated from the data repository of the source type to the data repository of the target type indicated by the target action.

2. The method of claim 1, wherein the plurality of action scores are determined using a trained reinforcement learning model.

3. The method according to claim 2, wherein the reinforcement learning model is trained through the following steps: Detect the first operational state of the virtual machine on the first type of data repository; Perform a first migration action to migrate the virtual machine from the first type of data repository to the second type of data repository; Detect the second operational state of the virtual machine on the second type of data repository; Determine the first reward for performing the first migration action; Based on the first operation state, the second operation state, the first reward, and the first migration action, a first action score is determined for performing the first migration action in the first operation state.

4. The method of claim 3, wherein determining the first reward comprises: Determine the first delay and the first throughput indicated by the first operating state; Detect the average latency and average throughput during the process of the virtual machine migrating from the first type of data repository to the second type of data repository; as well as The first reward is determined based on the difference between the first delay and the average delay, and the difference between the first throughput and the average throughput.

5. The method according to claim 3, further comprising: Determine multiple migration actions to migrate the virtual machine from the first type of data repository to the multiple types of data repositories respectively; Determine whether the first operation state is in the action scoring table used to maintain action scoring; In response to determining that the first operation state is not in the action scoring table, the first operation state is added to the action scoring table; as well as The first migration action is randomly selected from the multiple migration actions.

6. The method according to claim 5, further comprising: In response to determining the first operational state in the action scoring table, determine the probability of performing a random selection of a migration action; Based on the probability, determine whether to perform the random selection or the targeted selection based on the action rating table; as well as In response to determining to perform the random selection, a migration action is randomly selected from the plurality of migration actions as the first migration action.

7. The method according to claim 6, further comprising: In response to determining to perform the orientation selection, the migration action with the highest action score among the multiple migration actions is selected as the first migration action.

8. The method according to claim 3, further comprising: Obtain the training strategy, which includes the following items: a list of data repositories, the maximum number of migration actions to be performed, and the target performance range; and In response to determining that the second operating state is within the target performance range, the current training round ends; or In response to determining that the second operating state is not within the target performance range, it is determined whether the number of times the migration action has been performed has reached the maximum number; as well as In response to determining that the number of times has not reached the maximum number of times, the second operation state is updated to the first operation state.

9. The method of claim 3, wherein detecting the first operating state is triggered by receiving a migration request, and the migration request is sent by the virtual machine when performance degrades.

10. The method of claim 1, wherein the source operation status includes at least one of the following: virtual machine information, number of snapshots, type of data repository, workload, hardware settings, or operation performance metrics.

11. An electronic device, comprising: At least one processor; as well as At least one memory storing computer program instructions, the at least one memory and the computer program instructions being configured, together with the at least one processor, to cause the electronic device to perform actions, the actions including: Retrieve the source operation status of the virtual machine on the data repository of the source type; Identify multiple candidate migration actions to migrate the virtual machine from the source type of data repository to multiple types of data repositories; Based on the source operation state and the multiple candidate migration actions, multiple action scores are determined for the multiple candidate migration actions. The action scores indicate the operational performance of the virtual machine on the migrated data repository. Based on the multiple action scores, a target action is selected from the multiple candidate transfer actions; and By executing the target action, the virtual machine is migrated from the data repository of the source type to the data repository of the target type indicated by the target action.

12. The electronic device of claim 11, wherein the plurality of action scores are determined using a trained reinforcement learning model.

13. The electronic device of claim 12, wherein the reinforcement learning model is trained by the following steps: Detect the first operational state of the virtual machine on the first type of data repository; Perform a first migration action to migrate the virtual machine from the first type of data repository to the second type of data repository; Detect the second operational state of the virtual machine on the second type of data repository; Determine the first reward for performing the first migration action; Based on the first operation state, the second operation state, the first reward, and the first migration action, a first action score is determined for performing the first migration action in the first operation state.

14. The electronic device of claim 13, wherein determining the first reward comprises: Determine the first delay and the first throughput indicated by the first operating state; Detect the average latency and average throughput during the process of the virtual machine migrating from the first type of data repository to the second type of data repository; as well as The first reward is determined based on the difference between the first delay and the average delay, and the difference between the first throughput and the average throughput.

15. The electronic device according to claim 13, further comprising: Determine multiple migration actions to migrate the virtual machine from the first type of data repository to the multiple types of data repositories respectively; Determine whether the first operation state is in the action scoring table used to maintain action scoring; In response to determining that the first operation state is not in the action scoring table, the first operation state is added to the action scoring table; as well as The first migration action is randomly selected from the multiple migration actions.

16. The electronic device of claim 15, further comprising: In response to determining the first operational state in the action scoring table, determine the probability of performing a random selection of a migration action; Based on the probability, determine whether to perform the random selection or the targeted selection based on the action rating table; as well as In response to determining to perform the random selection, a migration action is randomly selected from the plurality of migration actions as the first migration action.

17. The electronic device of claim 16, further comprising: In response to determining to perform the orientation selection, the migration action with the highest action score among the multiple migration actions is selected as the first migration action.

18. The electronic device according to claim 13, further comprising: Obtain the training strategy, which includes the following items: a list of data repositories, the maximum number of migration actions to be performed, and the target performance range; and In response to determining that the second operating state is within the target performance range, the current training round ends; or In response to determining that the second operating state is not within the target performance range, it is determined whether the number of times the migration action has been performed has reached the maximum number; In response to determining that the number of times has not reached the maximum number of times, the second operation state is updated to the first operation state.

19. The electronic device of claim 11, wherein the plurality of data storage repositories comprises: Virtual Machine File System (VMFS), Virtual Volume (Vvol), or Virtual Storage Area Network (vSAN).

20. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a device to: Retrieve the source operation status of the virtual machine on the data repository of the source type; Identify multiple candidate migration actions to migrate the virtual machine from the source type of data repository to multiple types of data repositories; Based on the source operation state and the multiple candidate migration actions, multiple action scores are determined for the multiple candidate migration actions. The action scores indicate the operational performance of the virtual machine on the migrated data repository. Based on the multiple action scores, a target action is selected from the multiple candidate migration actions; as well as By executing the target action, the virtual machine is migrated from the data repository of the source type to the data repository of the target type indicated by the target action.

Images