Virtual machine migration method and device, computer equipment, storage medium and product

Abstract

The invention relates to a virtual machine migration method and device, computer equipment, a storage medium and a product. The method comprises the following steps: acquiring first performance information of an overload physical machine and second performance information of a virtual machine corresponding to the overload physical machine; inputting the first performance information and the second performance information into a pre-trained strategy network, and determining a to-be-migrated virtual machine; wherein the strategy network is obtained by training an initial deep reinforcement learning model according to a training sample, and the training sample comprises first sample information of a sample physical machine and second sample information of a sample virtual machine corresponding to the sample physical machine; and constructing a state vector, an action vector, an award and a next state vector according to the first sample information and the second sample information, wherein the award is determined according to the matching degree between the first sample information and the second sample information. According to the virtual machine migration method provided by the invention, the accuracy of determining the to-be-migrated virtual machine can be improved.

Description

Technical Field

[0001] This application relates to the field of virtualization technology, and in particular, to a virtual machine migration method, apparatus, computer device, storage medium, and product. Background Art

[0002] Virtualization technology allows a computer system to aggregate various resources, such as hardware resources, software resources, and data resources, and allocate appropriate resources according to user needs. In the complex environment of data center consolidation, the continuous changes in business requirements and resource utilization prompt virtual machines to migrate frequently between physical machines, which increases the difficulty of computer system management. Especially for the handling of overloaded physical machines, how to accurately select migration objects among numerous virtual machines to relieve the host pressure and maintain the overall stability of the data center has become an urgent problem to be solved.

[0003] In traditional technologies, by obtaining the resource requirements of virtual machines and the resources provided by physical machines, relevant algorithms are used to screen out the virtual machines that need to be migrated. However, the virtual machines screened out in this way are inaccurate. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a virtual machine migration method, apparatus, computer device, storage medium, and product that can improve the accuracy of determining the virtual machines that need to be migrated.

[0005] In a first aspect, this application provides a virtual machine migration method, which includes:

[0006] Obtain the first performance information of the overloaded physical machine and the second performance information of the virtual machines corresponding to the overloaded physical machine;

[0007] Input the first performance information and the second performance information into a pre-trained policy network to determine the virtual machines to be migrated;

[0008] Among them, the policy network is obtained by training an initial deep reinforcement learning model according to training samples. The training samples include the first sample information of sample physical machines and the second sample information of sample virtual machines corresponding to the sample physical machines. A state vector, an action vector, a reward, and a next state vector are constructed according to the first sample information and the second sample information, and the reward is determined according to the matching degree between the first sample information and the second sample information.

[0009] In one of the embodiments, the method for determining the reward according to the first sample information and the second sample information includes:

[0010] Obtain the first matching degree between the sample physical machine and each sample virtual machine according to the distance correlation coefficient between the first sample information and the second sample information;

[0011] Obtain the second matching degree between the sample physical machine and each sample virtual machine according to the resource supply-demand balance degree between the first sample information and the second sample information;

[0012] Determine the reward according to the first matching degree and / or the second matching degree.

[0013] In one embodiment, the initial deep reinforcement learning model includes an initial policy network and an initial value network, and the training method of the policy network includes:

[0014] Train the initial deep reinforcement learning model according to the training samples, adjust the network parameters of the initial policy network and the network parameters of the initial value network, and obtain the policy network.

[0015] In one embodiment, obtaining the first performance information of the overloaded physical machine and the second performance information of the virtual machine corresponding to the overloaded physical machine includes:

[0016] Obtain the first initial performance information of the overloaded physical machine and the second initial performance information of the virtual machine;

[0017] Predict the first performance information and the second performance information in a preset time period according to the first initial performance information and the second initial performance information.

[0018] In one embodiment, predicting the first performance information and the second performance information in a preset time period according to the first initial performance information and the second initial performance information includes:

[0019] Input the first initial performance information and the second initial performance information into a pre-trained prediction model, and predict the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information.

[0020] In one embodiment, the method further includes:

[0021] Smooth the first performance information and the second performance information respectively.

[0022] In a second aspect, an embodiment of the present application provides a virtual machine migration device, and the device includes:

[0023] An acquisition module, configured to acquire the first performance information of the overloaded physical machine and the second performance information of the virtual machine corresponding to the overloaded physical machine;

[0024] A determination module is configured to input first performance information and second performance information into a pre-trained policy network to determine a virtual machine to be migrated. The policy network is obtained by training an initial deep reinforcement learning model based on training samples. The training samples include first sample information of a sample physical machine and second sample information of a sample virtual machine corresponding to the sample physical machine. A state vector, an action vector, a reward, and a next state vector are constructed based on the first sample information and the second sample information. The reward is determined based on the matching degree between the first sample information and the second sample information.

[0025] In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method provided in the first aspect are implemented.

[0026] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0027] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0028] The above virtual machine migration method, device, computer device, storage medium and product. The method obtains the first performance information of an overloaded physical machine and the second performance information of the virtual machine corresponding to the overloaded physical machine; inputs the first performance information and the second performance information into a pre-trained policy network to determine the virtual machine to be migrated, and the policy network is obtained by training an initial deep reinforcement learning model based on training samples. The training samples include the first sample information of the sample physical machine and the second sample information of the sample virtual machine corresponding to the sample physical machine, and a state vector, an action vector, a reward and a next state vector are constructed according to the first sample information and the second sample information, and the reward is determined according to the matching degree between the first sample information and the second sample information. In this embodiment, determining the virtual machine to be migrated through the policy network in deep reinforcement learning can continuously learn and improve its policy during the interaction with the environment to maximize the long-term cumulative reward, can make a more flexible and intelligent migration policy in a complex cloud computing environment, can improve the accuracy of the determined virtual machine to be migrated, and thus can improve the overall efficiency and performance of the computer system. Moreover, the reward during the training of the policy network is determined according to the matching degree between the first sample information and the second sample information, which makes up for the deficiency of the quantization standard in evaluating the migration effect and provides a more scientific and reasonable decision-making basis for the dynamic scheduling and optimization of virtual machines in the cloud computing environment, thereby being able to improve the accuracy and reliability of using the policy network to determine the virtual machine to be migrated. In addition, the deep reinforcement learning module has strong generalization ability and can handle unseen states, which can improve the robustness and adaptability of the computer system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a schematic structural diagram of a computer device in an embodiment;

[0030] Figure 2 is a schematic flow chart of the steps of a virtual machine migration method in an embodiment;

[0031] Figure 3 is a schematic flow chart of the steps of a virtual machine migration method in another embodiment;

[0032] Figure 4 is a schematic structural diagram of a deep reinforcement learning model in an embodiment;

[0033] Figure 5 is a schematic flow chart of the steps of a virtual machine migration method in another embodiment;

[0034] Figure 6 is a schematic structural diagram of a GRU model in an embodiment;

[0035] Figure 7 is a schematic flow chart of the steps of a virtual machine migration method in another embodiment;

[0036] Figure 8 It is a schematic structural diagram of a virtual machine migration device in an embodiment. Specific implementation manners

[0037] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] The serial numbers assigned to the components in this article itself, such as "first", "second", etc., are only used to distinguish the described objects and do not have any sequential or technical meanings.

[0039] First, before specifically introducing the technical solutions of the disclosed embodiments of the present application, the background technology or the technical evolution context based on the embodiments of the present application will be introduced. As a new computing model, cloud computing technology has been widely applied in many fields. The cloud computing platform provides large-scale computing resources and storage capabilities, enabling users to use computing resources on demand without concerning about the underlying hardware facilities. Virtualization technology is one of the most important technologies in cloud computing. Virtualization technology allows a computer system to aggregate various resources, such as hardware resources, software resources, and data resources, and allocate appropriate resources according to user needs. As the core strategy for efficient integration of data center resources, the virtual machine migration technology process covers the precise determination of the migration timing, the careful selection of the target virtual machine, and the optimized configuration placement of the virtual machine in the new environment. Among these three major links, the proper selection of the virtual machine is particularly crucial. It not only deeply affects the energy consumption level of the data center but also directly relates to the optimization of resource utilization rate, which is a key step in balancing performance and cost. In the complex environment of data center integration, the continuous changes in business requirements and resource utilization rates prompt virtual machines to migrate frequently between physical machines to optimize resource configuration and achieve load balancing. However, this dynamic characteristic causes the virtual machine combinations on physical machines to change frequently, significantly increasing the difficulty of system management. Especially for the handling of overloaded physical machines, how to accurately select the migration object among numerous virtual machines to relieve the pressure on the physical machine and maintain the overall stability of the data center has become an urgent problem to be solved. This requires in-depth analysis of the performance requirements, resource consumption, and migration cost of each virtual machine and the construction of an efficient and accurate action space model to scientifically guide migration decisions and ensure that the operations are both reasonable and efficient.

[0040] In the prior art, physical machines can be sorted from high to low according to the degree of resource abundance, and the virtual machine with the least number of shared memory pages to be transferred during migration is preferentially selected for migration; the virtual machine can also be migrated by deeply integrating migration cost assessment and resource balance consideration; the virtual machine to be migrated can also be screened out through relevant algorithms by obtaining the resource requirements of the virtual machine and the resources provided by the physical machine. However, the virtual machines migrated using the prior art are inaccurate. For this reason, the present application provides a virtual machine migration method.

[0041] The virtual machine migration method provided by the present application can be applied to a computer device such as Figure 1 shown. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a virtual machine migration method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0042] Those skilled in the art can understand that Figure 1 the structure shown in

[0043] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0044] In one embodiment, as Figure 2 shown, a virtual machine migration method is provided. In this embodiment, the method is exemplified by being applied to a computer device. In this embodiment, the method includes the following steps:

[0045] Step 200: Obtain the first performance information of the overloaded physical machine and the second performance information of the virtual machine corresponding to the overloaded physical machine.

[0046] An overloaded physical machine refers to a physical machine (usually a hardware device such as a server) whose workload exceeds its designed rated capacity or normal operating range. The first performance information of the overloaded physical machine may include the Central Processing Unit (CPU) utilization rate and memory utilization rate of the overloaded physical machine within multiple consecutive equal preset time periods. The virtual machine corresponding to the overloaded physical machine refers to the virtual machine deployed on the overloaded physical machine, and multiple virtual machines can be deployed on one overloaded physical machine. The second performance information of the virtual machine may include the Central Processing Unit (CPU) utilization rate and memory utilization rate within multiple consecutive equal preset time periods. The first performance information of the overloaded physical machine and the second performance information of the virtual machine can be pre-stored in the memory of the computer device. When executing the virtual machine migration method, the computer device directly obtains the first performance information and the second performance information from the memory. This embodiment does not limit the specific method for obtaining the first performance information and the second performance information, as long as its function can be achieved.

[0047] In a specific embodiment, the CPU utilization rate of the overloaded physical machine in the first performance information can be expressed as , and the memory utilization rate of the overloaded physical machine can be expressed as , where p represents p consecutive equal preset time periods. For example, represents the CPU utilization rate of the overloaded physical machine within time periods, and represents the memory utilization rate of the overloaded physical machine in the th time period. The CPU utilization rate of the i-th virtual machine in the second performance information can be expressed as .

[0048] Step 210: Input the first performance information and the second performance information into a pre-trained policy network to determine the virtual machine to be migrated.

[0049] Among them, the policy network is obtained by training an initial deep reinforcement learning model according to training samples. The training samples include the first sample information of the sample physical machine and the second sample information of the sample virtual machine corresponding to the sample physical machine. A state vector, an action vector, a reward, and a next state vector are constructed according to the first sample information and the second sample information, and the reward is determined according to the matching degree between the first sample information and the second sample information.

[0050] The sample virtual machines are multiple virtual machines deployed on a sample physical machine, which can be an overloaded physical machine or a non-overloaded physical machine. The first sample information of the sample physical machine may include the CPU utilization rate of the sample physical machine and the memory utilization rate of the sample physical machine. The second sample information of the sample virtual machine may include the CPU utilization rate of the sample virtual machine and the memory utilization rate of the sample virtual machine.

[0051] After obtaining the first sample information and the second sample information, the computer device constructs a state vector, an action vector, a reward, and a next state vector according to the first sample information and the second sample information. The state vector refers to the performance information of the sample physical machine in the current time period and the performance information of each corresponding sample virtual machine. The action vector refers to selecting a sample virtual machine on the sample physical machine for migration. The reward is used to represent the reward obtained after performing the action in the current state. The next state vector refers to the state of the sample physical machine and each corresponding sample virtual machine after performing the action. Among them, the reward is determined by the matching degree between the first sample information and the second sample information. That is to say, by determining the matching degree between the performance of the sample physical machine and the performance of each sample virtual machine after performing the action, the reward in the training sample is determined. The higher the matching degree, the higher the reward.

[0052] The computer device trains the initial deep reinforcement learning model according to the training samples, that is, trains the initial policy network and the initial value network in the initial deep reinforcement learning model, and can train to obtain a policy network and a value network. The specific process of training the initial deep reinforcement learning model according to the training samples in this embodiment is not limited as long as its function can be realized.

[0053] The policy network can be pre-trained and stored in the computer device. After obtaining the first performance information and the second performance information, the computer device can construct a state vector according to the first performance information and the second performance information, and input the state vector into the policy network to obtain a corresponding action vector. That is to say, it can determine the virtual machine to be migrated, that is, the virtual machine to be migrated.

[0054] An embodiment of the present application provides a virtual machine migration method. The method includes obtaining first performance information of an overloaded physical machine and second performance information of a virtual machine corresponding to the overloaded physical machine; inputting the first performance information and the second performance information into a pre-trained policy network to determine a virtual machine to be migrated, where the policy network is obtained by training an initial deep reinforcement learning model according to training samples; the training samples include first sample information of a sample physical machine and second sample information of a sample virtual machine corresponding to the sample physical machine, and a state vector, an action vector, a reward, and a next state vector are constructed according to the first sample information and the second sample information, and the reward is determined according to the matching degree between the first sample information and the second sample information. In this embodiment, by using the policy network in deep reinforcement learning to determine the virtual machine to be migrated, it can continuously learn and improve its policy during the interaction with the environment to maximize the long-term cumulative reward, make a more flexible and intelligent migration policy in a complex cloud computing environment, improve the accuracy of the determined virtual machine to be migrated, and thus improve the overall efficiency and performance of the computer system. Moreover, the reward during the training of the policy network is determined according to the matching degree between the first sample information and the second sample information, which makes up for the deficiency of the quantization standard in evaluating the migration effect, provides a more scientific and reasonable decision-making basis for the dynamic scheduling and optimization of virtual machines in the cloud computing environment, and thus can improve the accuracy and reliability of using the policy network to determine the virtual machine to be migrated. In addition, the deep reinforcement learning model has strong generalization ability and can handle unseen states, which can improve the robustness and adaptability of the computer system.

[0055] In one embodiment, as Figure 3 shown, it relates to an implementation method for determining a reward according to first sample information and second sample information. The steps of the implementation method include:

[0056] Step 300: Obtain a first matching degree between the sample physical machine and each sample virtual machine according to the distance correlation coefficient between the first sample information and the second sample information.

[0057] After the computer device obtains the first sample information of the sample physical machine and the second sample information of each corresponding sample virtual machine, by calculating the distance correlation coefficient between the first sample information and the second sample information, the adaptability between the sample physical machine and each sample virtual machine, that is, the first matching degree, can be obtained. The higher the first matching degree, the more the resources of the sample physical machine tend to be in an equilibrium state after migrating the sample virtual machine.

[0058] In an optional embodiment, the first sample information of the sample physical machine can be expressed as: , and the second sample information of the i-th sample virtual machine can be expressed as The first matching degree between the sample physical machine and the i-th sample virtual machine, that is, the distance correlation coefficient between the first sample information and the second sample information, can be expressed as: , where represents the distance covariance between and represents the distance variance of X, represents the distance variance of. The distance covariance can be calculated using the formula , where n is the dimension of X and Y, , represents the mean of the j-th column, represents the mean of the i-th row in X, represents the mean of all elements in X. In the same way, can be calculated. The distance variance can be calculated using the formula , and in the same way, can be calculated.

[0059] Step 310: Obtain the second matching degree between the sample physical machine and each sample virtual machine according to the resource supply-demand balance degree between the first sample information and the second sample information.

[0060] After the computer device obtains the first sample information of the sample physical machine and the corresponding second sample information of each sample virtual machine, by determining the resource supply-demand balance degree between the first sample information and the second sample information, that is, the matching degree between the available resources of the sample physical machine and the resource requirements of the sample virtual machine, the second matching degree between the sample physical machine and each sample virtual machine can be obtained. Specifically, the second matching degree between the sample physical machine and each sample virtual machine includes the CPU supply-demand balance degree between the sample physical machine and each sample virtual machine, and the memory supply-demand balance degree between the sample physical machine and each sample virtual machine. The lower the second matching degree, the more the sample physical machine tends to be in resource supply-demand balance after the sample virtual machine is migrated.

[0061] In a specific embodiment, the second matching degree between the sample physical machine and the i-th sample virtual machine can be expressed as: , where represents the CPU supply-demand balance degree between the i-th sample virtual machine and the sample physical machine at the k-th moment, represents the memory supply-demand balance degree between the i-th sample virtual machine and the sample physical machine at the k-th moment, is the importance parameter of the CPU, is the importance parameter of the memory, . Among them, , , C represents the total CPU of the sample physical machine, represents the total CPU of the i-th sample virtual machine, M represents the total memory of the sample physical machine, represents the total memory of the i sample virtual machines.

[0062] Step 320, determine the reward according to the first matching degree and / or the second matching degree.

[0063] After obtaining the first matching degree and the second matching degree, the computer device determines the reward according to the first matching degree and / or the second matching degree. That is to say, the computer device can determine the first matching degree as the reward, that is , or determine the second matching degree as the reward, that is, , or determine the reward by calculating the difference between the first matching degree and the second matching degree, that is .

[0064] In this embodiment, the first matching degree between the sample physical machine and each sample virtual machine is obtained according to the distance correlation coefficient between the first sample information and the second sample information; the second matching degree between the sample physical machine and the sample virtual machine is obtained according to the resource supply-demand balance degree between the first sample information and the second sample information; the reward is determined according to the first matching degree and / or the second matching degree. The first matching degree can take into account the balance state of the sample physical machine after migrating the sample virtual machine, and the second matching degree can take into account the balance between the resource requirements of the sample virtual machine and the available resources of the sample physical machine. Using the reward determined in this way as an important indicator for training the model can improve the accuracy of the virtual machine to be migrated determined by the trained policy network.

[0065] In one embodiment, the initial deep reinforcement learning model includes an initial policy network and an initial value network. The training method of the policy network includes:

[0066] Train the initial deep reinforcement learning model according to the training samples, and adjust the network parameters of the initial policy network and the network parameters of the initial value network to obtain the policy network.

[0067] The state vector, action vector, reward, and next state vector constructed from the training samples are stored in the experience replay pool. When the computer device trains the initial deep reinforcement learning model, it obtains the state vector from the experience replay pool, inputs the state vector into the initial policy network, and outputs a probability distribution of an action; inputs the state vector into the initial value network, and outputs the reward corresponding to the state vector. The network parameters of the initial policy network are updated by calculating the policy gradient, and the network parameters of the initial value network are updated by calculating the loss function of the value network. The above training process is repeated to continuously optimize the network parameters of the initial policy network and the network parameters of the initial value network until the preset stop condition is reached, and a policy network is obtained. Among them, the preset stop condition can be that the number of training times reaches a certain threshold or the cumulative reward no longer increases significantly.

[0068] In this embodiment, the initial deep reinforcement learning model is trained using the training samples to adjust the network parameters of the initial policy network and the initial value network, and a policy network is obtained. This method of training the deep reinforcement learning model is fast and easy to implement.

[0069] In an alternative embodiment, the structural diagram of the deep reinforcement learning model is as Figure 4 shown. Among them, the neural network includes a Convolutional Neural Network (CNN) and a Fully Connected Layer (FC). The neural network provides a basic implementation framework for the policy network and the value network in the deep reinforcement learning model. The policy network and the value network cooperate with each other functionally to jointly complete the decision-making and learning tasks of the intelligent agent. By introducing a target network to assist in the training of the value network, the stability of the deep reinforcement learning model can be improved.

[0070] In one embodiment, as Figure 5 shown, it relates to an implementation method for obtaining overloaded physical machine information and virtual machine information. The steps of this implementation method include:

[0071] Step 500, obtain the first initial performance information of the overloaded physical machine and the second initial performance information of the virtual machine.

[0072] The descriptions of the overloaded physical machine and the virtual machine can refer to the specific descriptions of the above embodiments and will not be elaborated here. The first initial performance information may be the current CPU utilization rate and memory utilization rate of the overloaded physical machine, and the second initial performance information is the CPU utilization rate and memory utilization rate of multiple virtual machines deployed on the current overloaded physical machine. The first initial performance information and the second initial performance information can be used to determine the performance of the overloaded physical machine and the corresponding multiple virtual machines for acquisition.

[0073] Step 510: Predict the first performance information and the second performance information for a preset time period based on the first initial performance information and the second initial performance information.

[0074] After the computer device obtains the first initial performance information, it makes a prediction based on the first initial performance information to obtain the first performance information within a future preset time period; after obtaining the second initial performance information, it makes a prediction based on the second initial performance information to obtain the second performance information within a future preset time period. The preset time period may be the same as or different from the time period for obtaining the first initial performance information and the second initial performance information. In this embodiment, there is no limitation on the specific method for predicting the first performance information and the second performance information for the preset time period based on the first initial performance information and the second initial performance information, as long as its function can be realized.

[0075] In this embodiment, by obtaining the first initial performance information of the overloaded physical machine and the second initial performance information of the virtual machine, and predicting the first performance information and the second performance information for a preset time period based on the first initial performance information and the second initial performance information, considering that the performance of the physical machine and the virtual machine is dynamically variable and not fixed, predicting the obtained first initial performance information and second initial performance information to obtain the first performance information and the second performance information can improve the accuracy of determining the virtual machine to be migrated based on the first performance information and the second performance information.

[0076] In one embodiment, an implementation manner for predicting the first performance information and the second performance information for a preset time period based on the first initial performance information and the second initial performance information is involved. This implementation manner includes:

[0077] Input the first initial performance information and the second initial performance information into a pre-trained prediction model, and predict the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information.

[0078] The prediction model can be obtained by training an initial network model with training samples of performance information and pre-stored in a computer device. The prediction model can be a Gated Recurrent Unit (GRU) model. The GRU model includes an update gate and a reset gate. The update gate is responsible for adjusting the influence degree of historical information on the current state. It can utilize the non-linear characteristics of the sigmoid function to intelligently determine which historical information should be retained to assist future predictions. That is to say, through the update gate, the information in the first initial performance information and the second initial performance information of the input that can be used for prediction can be determined. This can avoid the problem of gradient disappearance or explosion existing in traditional recurrent neural networks, enabling the model to process long sequence data more stably. The reset gate can be used for screening. It uses the output of the sigmoid function to determine which historical information should be ignored or "reset". This can filter out irrelevant or redundant information, helping the model focus on the information most valuable to the current prediction task. In other words, the GRU model has the ability to process long sequence data and can effectively capture the deep associations and dynamic changes between CPU (memory) usage patterns in a complex and changing virtual environment. Using the GRU model for prediction can obtain more accurate first performance information and second performance information based on comprehensive historical data and current context information.

[0079] In an optional embodiment, the structure of the GRU model is as Figure 6 shown. Among them, represents the input vector at the current time step t (the first initial performance information in the t-th time period), represents the hidden state at the previous time step t - 1, represents the sigmoid function, represents the information to be forgotten in the hidden state at the previous time step t - 1, represents the information in the hidden state at the previous time step t - 1 that needs to be passed to the information of, represents the hidden state at the current moment. A " " in the circle represents element-wise multiplication of vectors, a "+" in the non-shaded circle represents vector sum, a "+" in the shaded circle represents vector concatenation, tanh represents the hyperbolic tangent function, represents the candidate hidden state.

[0080] After obtaining the first initial performance information and the second initial performance information, the computer device inputs the first initial performance information and the second initial performance information into a pre-trained prediction model, and can predict the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information. Specifically, the computer device can first input the first initial performance information into the prediction model to predict the corresponding first performance information, and then input the second initial performance information into the prediction model to predict the corresponding second performance information. It can also input the first initial performance information and the second initial performance information into the prediction model at the same time, and the prediction model respectively predicts the first initial performance information and the second initial performance information to obtain the corresponding first performance information and second performance information. This embodiment does not limit this.

[0081] In a specific embodiment, taking the CPU utilization rate in multiple consecutive equal time periods in the first initial performance information of the overloaded physical machine as an example, the CPU utilization rate is input into the GRU model. After the cyclic processing of the GRU model, the hidden state at each time step will capture the temporal dependence relationship in the CPU utilization rate sequence at the current time step and the previous time steps. In this way, the hidden state not only contains the information at the current time point, but also integrates the context information in the historical data, enabling the GRU model to more accurately understand the dynamic changes of the CPU utilization rate, and finally output the predicted first performance information.

[0082] In this embodiment, by inputting the first initial performance information and the second initial performance information into the pre-trained prediction model, the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information are predicted, which can improve the accuracy and efficiency of determining the first performance information and the second performance information.

[0083] In one embodiment, during the process of training the policy network, the training samples obtained can also be the training samples after prediction.

[0084] In one embodiment, after obtaining the first performance information and the second performance information, the method further includes:

[0085] Smoothing the first performance information and the second performance information respectively.

[0086] After obtaining the first performance information of the overloaded physical machine, the computer device performs smoothing processing on the first performance information to weaken the abnormal information in the first performance information. After obtaining the second performance information of each virtual machine, the computer device performs smoothing processing on the second performance information to weaken the abnormal information in the second performance information. The smoothing processing methods that can be adopted include exponential smoothing method, wavelet transform method, moving average method, filtering method, etc. This embodiment does not limit this, as long as its function can be realized. The computer device inputs the smoothed first performance information and second performance information into a pre-trained policy network to obtain the virtual machines to be migrated.

[0087] In an optional embodiment, taking the example of using the exponential smoothing method to smooth the CPU utilization rate of the predicted overloaded physical machine, the CPU utilization rate of the overloaded physical machine after smoothing can be expressed as: , where is the smoothing constant, and its value is between 0 and 1. represents the CPU utilization rate of the overloaded physical machine at the j-th moment obtained by prediction. represents the CPU utilization rate of the overloaded physical machine after smoothing at the (j - 1)-th moment. represents the CPU utilization rate of the overloaded physical machine after smoothing at the j-th moment.

[0088] In this embodiment, by respectively performing smoothing processing on the first performance information and the second performance information, it is possible to reduce the adverse impact of the large-fluctuation data in the predicted first performance information and second performance information on the finally determined virtual machines to be migrated, thereby improving the accuracy of the finally determined virtual machines to be migrated.

[0089] Please refer to Figure 7 , an embodiment of the present application provides a virtual machine migration method, and the steps of this method include:

[0090] Step 700, obtain training samples, where the training samples include the first sample information of the sample physical machine and the second sample information of the sample virtual machine corresponding to the sample physical machine;

[0091] Step 710, obtain the first matching degree between the sample physical machine and each sample virtual machine according to the distance correlation coefficient between the first sample information and the second sample information;

[0092] Step 720, obtain the second matching degree between the sample physical machine and each sample virtual machine according to the resource supply-demand balance degree between the first sample information and the second sample information;

[0093] Step 730, determine the reward in the training samples according to the first matching degree and the second matching degree;

[0094] Step 740: Construct a state vector, an action vector, a reward, and a next state vector based on the first sample information and the second sample information, and store the state vector, the action vector, the reward, and the next state vector in the experience replay pool;

[0095] Step 750: Collect sub-training samples from the experience replay pool, and use the sub-training samples to train the initial deep reinforcement learning model, and adjust the network parameters of the initial policy network and the network parameters of the initial value network to obtain the policy network;

[0096] Step 760: Obtain the first initial performance information of the overloaded physical machine and the first initial performance information of each corresponding virtual machine;

[0097] Step 770: Input the first initial performance information and the second initial performance information into a pre-trained GRU model to obtain the first performance information and the second performance information;

[0098] Step 780: Smooth the first performance information and the second performance information respectively;

[0099] Step 790: Input the smoothed first performance information and the smoothed second performance information into the policy network to obtain the virtual machines to be migrated.

[0100] It should be understood that although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the arrows, these steps do not necessarily need to be executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps does not have a strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily need to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily need to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0101] Based on the same inventive concept, an embodiment of the present application further provides a virtual machine migration device for implementing the above-mentioned virtual machine migration method. The implementation solution provided by this device to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following virtual machine migration device can refer to the limitations on the virtual machine migration method in the above text, and will not be repeated here.

[0102] In one embodiment, as Figure 8 shown, a virtual machine migration device 10 is provided, including: an acquisition module 11 and a determination module 12, where:

[0103] An obtaining module 11, configured to obtain first performance information of an overloaded physical machine and second performance information of a virtual machine corresponding to the overloaded physical machine.

[0104] A determining module 12, configured to input the first performance information and the second performance information into a pre-trained policy network to determine a virtual machine to be migrated; wherein, the policy network is obtained by training an initial deep reinforcement learning model according to training samples, the training samples include first sample information of a sample physical machine and second sample information of a sample virtual machine corresponding to the sample physical machine, a state vector, an action vector, a reward, and a next state vector are constructed according to the first sample information and the second sample information, and the reward is determined according to a matching degree between the first sample information and the second sample information.

[0105] In one embodiment, the determining module 12 is further configured to obtain a first matching degree between the sample physical machine and each sample virtual machine according to a distance correlation coefficient between the first sample information and the second sample information; obtain a second matching degree between the sample physical machine and each sample virtual machine according to a resource supply-demand balance degree between the first sample information and the second sample information; and determine the reward according to the first matching degree and / or the second matching degree.

[0106] In one embodiment, the virtual machine migration device 10 further includes a training module. The training module is configured to train the initial deep reinforcement learning model according to the training samples, and adjust network parameters of the initial policy network and network parameters of the initial value network to obtain the policy network.

[0107] In one embodiment, the obtaining module 11 includes an obtaining unit and a predicting unit. The obtaining unit is configured to obtain first initial performance information of the overloaded physical machine and second initial performance information of the virtual machine; the predicting unit is configured to predict the first performance information and the second performance information in a preset time period according to the first initial performance information and the second initial performance information.

[0108] In one embodiment, the predicting unit is specifically configured to input the first initial performance information and the second initial performance information into a pre-trained prediction model, and predict the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information.

[0109] In one embodiment, the virtual machine migration device 10 further includes a processing module. The processing module is configured to perform smoothing processing on the first performance information and the second performance information respectively.

[0110] Each module in the above virtual machine migration device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0111] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0112] Obtain the first performance information of the overloaded physical machine and the second performance information of the virtual machine corresponding to the overloaded physical machine;

[0113] Input the first performance information and the second performance information into a pre-trained policy network to determine the virtual machine to be migrated;

[0114] Among them, the policy network is obtained by training an initial deep reinforcement learning model according to training samples. The training samples include the first sample information of the sample physical machine and the second sample information of the sample virtual machine corresponding to the sample physical machine. A state vector, an action vector, a reward, and a next state vector are constructed according to the first sample information and the second sample information. The reward is determined according to the matching degree between the first sample information and the second sample information.

[0115] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Obtain the first matching degree between the sample physical machine and each sample virtual machine according to the distance correlation coefficient between the first sample information and the second sample information; Obtain the second matching degree between the sample physical machine and each sample virtual machine according to the resource supply-demand balance degree between the first sample information and the second sample information; Determine the reward according to the first matching degree and / or the second matching degree.

[0116] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Train the initial deep reinforcement learning model according to the training samples, and adjust the network parameters of the initial policy network and the network parameters of the initial value network to obtain the policy network.

[0117] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Obtain the first initial performance information of the overloaded physical machine and the second initial performance information of the virtual machine; Predict the first performance information and the second performance information in a preset time period according to the first initial performance information and the second initial performance information.

[0118] In one embodiment, when the processor executes the computer program, the following steps are further implemented: input the first initial performance information and the second initial performance information into a pre-trained prediction model, and predict the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information.

[0119] In one embodiment, when the processor executes the computer program, the following steps are further implemented: perform smoothing processing on the first performance information and the second performance information respectively.

[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0121] Obtain the first performance information of the overloaded physical machine and the second performance information of the virtual machine corresponding to the overloaded physical machine;

[0122] Input the first performance information and the second performance information into a pre-trained policy network to determine the virtual machine to be migrated;

[0123] Wherein, the policy network is obtained by training an initial deep reinforcement learning model according to training samples. The training samples include the first sample information of the sample physical machine and the second sample information of the sample virtual machine corresponding to the sample physical machine. A state vector, an action vector, a reward, and a next state vector are constructed according to the first sample information and the second sample information, and the reward is determined according to the matching degree between the first sample information and the second sample information.

[0124] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: obtain the first matching degree between the sample physical machine and each sample virtual machine according to the distance correlation coefficient between the first sample information and the second sample information; obtain the second matching degree between the sample physical machine and each sample virtual machine according to the resource supply-demand balance degree between the first sample information and the second sample information; determine the reward according to the first matching degree and / or the second matching degree.

[0125] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: train the initial deep reinforcement learning model according to the training samples, and adjust the network parameters of the initial policy network and the network parameters of the initial value network to obtain the policy network.

[0126] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: obtain the first initial performance information of the overloaded physical machine and the second initial performance information of the virtual machine; predict the first performance information and the second performance information in a preset time period according to the first initial performance information and the second initial performance information.

[0127] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: inputting the first initial performance information and the second initial performance information into a pre-trained prediction model, and predicting the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information.

[0128] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: respectively performing smoothing processing on the first performance information and the second performance information.

[0129] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0130] Obtaining the first performance information of an overloaded physical machine and the second performance information of a virtual machine corresponding to the overloaded physical machine;

[0131] Inputting the first performance information and the second performance information into a pre-trained policy network to determine a virtual machine to be migrated;

[0132] Wherein, the policy network is obtained by training an initial deep reinforcement learning model according to training samples, the training samples include the first sample information of a sample physical machine and the second sample information of a sample virtual machine corresponding to the sample physical machine, constructing a state vector, an action vector, a reward, and a next state vector according to the first sample information and the second sample information, and the reward is determined according to the matching degree between the first sample information and the second sample information.

[0133] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining a first matching degree between the sample physical machine and each sample virtual machine according to the distance correlation coefficient between the first sample information and the second sample information; obtaining a second matching degree between the sample physical machine and each sample virtual machine according to the resource supply-demand balance degree between the first sample information and the second sample information; determining the reward according to the first matching degree and / or the second matching degree.

[0134] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: training the initial deep reinforcement learning model according to the training samples, and adjusting the network parameters of the initial policy network and the network parameters of the initial value network to obtain the policy network.

[0135] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining the first initial performance information of an overloaded physical machine and the second initial performance information of a virtual machine; predicting the first performance information and the second performance information in a preset time period according to the first initial performance information and the second initial performance information.

[0136] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: inputting the first initial performance information and the second initial performance information into a pre-trained prediction model to predict the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information.

[0137] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: respectively performing smoothing processing on the first performance information and the second performance information.

[0138] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, a database, or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAMs), magnetoresistive random access memories (MRAMs), ferroelectric random access memories (FRAMs), phase change memories (PCMs), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0139] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0140] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A virtual machine migration method, characterized in that, The method includes: Obtaining first performance information of an overloaded physical machine and second performance information of a virtual machine corresponding to the overloaded physical machine; Inputting the first performance information and the second performance information into a pre-trained policy network to determine a virtual machine to be migrated; Wherein, the policy network is obtained by training an initial deep reinforcement learning model according to training samples, the training samples include first sample information of a sample physical machine and second sample information of a sample virtual machine corresponding to the sample physical machine, a state vector, an action vector, a reward, and a next state vector are constructed according to the first sample information and the second sample information, and the reward is determined according to the matching degree between the first sample information and the second sample information.

2. The method according to claim 1, wherein The method for determining the reward according to the first sample information and the second sample information includes: Obtaining a first matching degree between the sample physical machine and each of the sample virtual machines according to a distance correlation coefficient between the first sample information and the second sample information; Obtaining a second matching degree between the sample physical machine and each of the sample virtual machines according to a resource supply-demand balance degree between the first sample information and the second sample information; Determining the reward according to the first matching degree and / or the second matching degree.

3. The method according to claim 1 or 2, characterized in that, The initial deep reinforcement learning model includes an initial policy network and an initial value network, and the training method of the policy network includes: Training the initial deep reinforcement learning model according to the training samples, and adjusting network parameters of the initial policy network and network parameters of the initial value network to obtain the policy network.

4. The method according to claim 1 or 2, characterized in that, The obtaining of the first performance information of the overloaded physical machine and the second performance information of the virtual machine corresponding to the overloaded physical machine includes: Obtaining first initial performance information of the overloaded physical machine and second initial performance information of the virtual machine; Predicting the first performance information and the second performance information in a preset time period according to the first initial performance information and the second initial performance information.

5. The method according to claim 4, wherein The predicting of the first performance information and the second performance information in a preset time period according to the first initial performance information and the second initial performance information includes: Inputting the first initial performance information and the second initial performance information into a pre-trained prediction model to predict the first performance information corresponding to the first initial performance information and the second performance information corresponding to the second initial performance information.

6. The method according to claim 5, wherein The method further includes: Smoothing the first performance information and the second performance information respectively.

7. A virtual machine migration device, characterized in that, The device includes: An obtaining module, configured to obtain first performance information of an overloaded physical machine and second performance information of a virtual machine corresponding to the overloaded physical machine; A determination module, configured to input the first performance information and the second performance information into a pre-trained policy network to determine a virtual machine to be migrated; wherein the policy network is obtained by training an initial deep reinforcement learning model according to training samples, the training samples include first sample information of a sample physical machine and second sample information of a sample virtual machine corresponding to the sample physical machine, a state vector, an action vector, a reward, and a next state vector are constructed according to the first sample information and the second sample information, and the reward is determined according to a matching degree between the first sample information and the second sample information.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1-6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images