Virtual machine deployment method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a virtual machine deployment method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring first physical machine resource information of a plurality of physical machines in a future time period when virtual machines are not deployed, and virtual machine resource information of a plurality of virtual machines in the future time period when the virtual machines are not deployed to the physical machines; predicting second physical machine resource information of the plurality of physical machines in a future time period under the condition that the plurality of virtual machines are deployed on the plurality of physical machines based on the virtual machine deployment information; according to the second physical machine resource information of the plurality of physical machines in the future time period, environment resource evaluation information of the cloud computing environment is detected; based on deployment strategy reward information generated by the environment resource evaluation information, performing iterative updating on the virtual machine deployment information to obtain target virtual machine deployment information; and deploying the plurality of virtual machines to the plurality of physical machines according to the target virtual machine deployment information. By adopting the method, the virtual machine can be accurately deployed.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a virtual machine deployment method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of cloud computing technology, virtualization technology has emerged. As the core foundation of cloud computing, virtualization technology provides the possibility of simulating the deployment of multiple independent computer systems (i.e. virtual machines) on a single physical machine. In a cloud computing environment, the deployment of virtual machines is a key technology to achieve more flexible data center resource integration. By deploying virtual machines on different physical machines, physical servers can be maintained, upgraded or reconfigured without affecting business continuity, thereby optimizing resource utilization.

[0003] At present, the essence of virtual machine deployment technology is to find the most suitable physical machine for the currently deployed virtual machine. At present, common methods of virtual machine deployment include: when processing the deployment request of the virtual machine, first evaluate the remaining resources of each physical machine to determine its ability to accommodate the virtual machine, and generate multiple initial virtual machine deployment scenarios based on the remaining resources of each physical machine. Then, all initial virtual machine deployment scenarios are regarded as particles in the particle swarm optimization algorithm, each particle represents a deployment strategy, and the optimal deployment plan of the virtual machine is finally determined by iteratively adjusting the position and speed of each particle and utilizing the intelligent optimization characteristics of the particle swarm optimization algorithm.

[0004] However, this virtual machine deployment method still has the problem of being inaccurate. Summary of the invention

[0005] Based on this, it is necessary to provide an accurate virtual machine deployment method, apparatus, computer equipment, computer-readable storage medium and computer program product to address the above technical issues.

[0006] In a first aspect, the present application provides a virtual machine deployment method, comprising:

[0007] Generate initialized virtual machine deployment information, wherein the virtual machine deployment information is used to indicate that multiple virtual machines are deployed to multiple physical machines in a cloud computing environment;

[0008] Obtaining first physical machine resource information in a future time period when multiple physical machines have not deployed virtual machines, and virtual machine resource information in a future time period when multiple virtual machines have not been deployed to physical machines;

[0009] Prediction step: predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information;

[0010] Detecting environmental resource evaluation information of the cloud computing environment according to resource information of a second physical machine of the multiple physical machines in a future time period, wherein the environmental resource evaluation information represents at least one of a resource balance and energy consumption of the cloud computing environment;

[0011] Iteratively update the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information to obtain the target virtual machine deployment information;

[0012] Deploy multiple virtual machines to multiple physical machines based on target virtual machine deployment information.

[0013] In one embodiment, based on the first physical machine resource information and the virtual machine resource information, predicting the second physical machine resource information of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information includes:

[0014] Obtaining a deployment time step for deploying a virtual machine in the virtual machine deployment information, and determining a future time period corresponding to the deployment time step, wherein the future time period includes multiple time steps;

[0015] Based on the first physical machine resource information and the virtual machine resource information at each time step, predicting the second physical machine resource information of the multiple physical machines at each time step when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information;

[0016] Based on the second physical machine resource information of the plurality of physical machines at each time step, the second physical machine resource information of the plurality of physical machines in a future time period is predicted.

[0017] In one embodiment, the virtual machine resource information includes the effective use time of the virtual machine processor and the memory usage of the virtual machine; the first physical machine resource information includes the effective use time of the physical machine processor and the memory usage of the physical machine, and the second physical machine resource information includes the processor usage rate and the memory usage rate;

[0018] Predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information, includes:

[0019] According to the effective use time of the processor of the physical machine and the effective use time of the processor of the virtual machine, predict the processor usage rates of the multiple physical machines in the future time period when multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information;

[0020] According to the memory usage of the physical machine and the memory usage of the virtual machine, the memory usage of the multiple physical machines in the future time period is predicted when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information.

[0021] In one embodiment, predicting the processor usage rates of multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on virtual machine deployment information according to the effective use time of the physical machine processor and the effective use time of the virtual machine processor includes:

[0022] For any target physical machine among the multiple physical machines, obtain the total execution time of the processor of the target physical machine;

[0023] According to the total processor execution time, the effective use time of the physical machine processor of the target physical machine, and the effective use time of the virtual machine processor of each virtual machine deployed on the target physical machine, predict the processor usage rate of the target physical machine in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information;

[0024] Based on the memory usage of the physical machine and the memory usage of the virtual machine, the memory usage of multiple physical machines in the future time period is predicted when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information, including:

[0025] For any target physical machine among the multiple physical machines, obtain the total memory execution time of the target physical machine;

[0026] Based on the total memory execution time, the physical machine memory usage of the target physical machine, and the virtual machine memory usage of each virtual machine deployed on the target physical machine, the memory usage of the target physical machine in the future time period is predicted when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information.

[0027] In one embodiment, the environmental resource evaluation information represents the resource balance and energy consumption of the cloud computing environment, and detecting the environmental resource evaluation information of the cloud computing environment according to the second physical machine resource information of the multiple physical machines in the future time period includes:

[0028] Obtain the number of physical machines of multiple physical machines, the full-load energy consumption of the physical machines when the multiple physical machines are in a fully loaded state, and the basic energy consumption of the physical machines when the multiple physical machines are in an idle state;

[0029] Detect resource balance of cloud computing environment based on processor utilization, memory utilization and number of physical machines of multiple physical machines in future time periods;

[0030] The energy consumption of the cloud computing environment is detected according to the processor utilization rates of the multiple physical machines in the future time period, the full load energy consumption of the multiple physical machines in the full load state, and the basic energy consumption of the multiple physical machines in the idle state.

[0031] In one embodiment, the virtual machine deployment information is iteratively updated based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information to obtain the target virtual machine deployment information, including:

[0032] Generate deployment strategy reward information corresponding to virtual machine deployment information based on environmental resource evaluation information;

[0033] Based on the deployment strategy reward information, the first physical machine resource information and the virtual machine resource information are updated, and based on the updated first physical machine resource information and the virtual machine resource information, the virtual machine deployment information is updated;

[0034] Return to the prediction step until the deployment strategy reward information converges or the number of iterations reaches a preset threshold, and select the target virtual machine deployment information from the virtual machine deployment information updated during the multiple rounds of iterations based on the deployment strategy reward information generated during the multiple rounds of iterations.

[0035] In a second aspect, the present application also provides a virtual machine deployment device, including:

[0036] A deployment information generation module, used to generate initialized virtual machine deployment information, wherein the virtual machine deployment information is used to indicate the deployment of multiple virtual machines to multiple physical machines in a cloud computing environment;

[0037] An initial resource information acquisition module, used to acquire resource information of a first physical machine in a future time period when multiple physical machines have not deployed virtual machines, and resource information of virtual machines in a future time period when multiple virtual machines have not been deployed to physical machines;

[0038] A prediction module, used for the prediction step: predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information;

[0039] An environmental resource evaluation module, configured to detect environmental resource evaluation information of a cloud computing environment based on resource information of a second physical machine of the plurality of physical machines in a future time period, wherein the environmental resource evaluation information represents at least one of a resource balance and energy consumption of the cloud computing environment;

[0040] An iterative processing module, configured to iteratively update the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information, and obtain the target virtual machine deployment information;

[0041] The deployment module is used to deploy multiple virtual machines to multiple physical machines according to the target virtual machine deployment information.

[0042] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0043] Generate initialized virtual machine deployment information, wherein the virtual machine deployment information is used to indicate that multiple virtual machines are deployed to multiple physical machines in a cloud computing environment;

[0044] Obtaining first physical machine resource information in a future time period when multiple physical machines have not deployed virtual machines, and virtual machine resource information in a future time period when multiple virtual machines have not been deployed to physical machines;

[0045] Prediction step: predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information;

[0046] Detecting environmental resource evaluation information of the cloud computing environment according to resource information of a second physical machine of the multiple physical machines in a future time period, wherein the environmental resource evaluation information represents at least one of a resource balance and energy consumption of the cloud computing environment;

[0047] Iteratively update the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information to obtain the target virtual machine deployment information;

[0048] Deploy multiple virtual machines to multiple physical machines based on target virtual machine deployment information.

[0049] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0050] Generate initialized virtual machine deployment information, wherein the virtual machine deployment information is used to indicate that multiple virtual machines are deployed to multiple physical machines in a cloud computing environment;

[0051] Obtaining first physical machine resource information in a future time period when multiple physical machines have not deployed virtual machines, and virtual machine resource information in a future time period when multiple virtual machines have not been deployed to physical machines;

[0052] Prediction step: predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information;

[0053] Detecting environmental resource evaluation information of the cloud computing environment according to resource information of a second physical machine of the multiple physical machines in a future time period, wherein the environmental resource evaluation information represents at least one of a resource balance and energy consumption of the cloud computing environment;

[0054] Iteratively update the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information to obtain the target virtual machine deployment information;

[0055] Deploy multiple virtual machines to multiple physical machines based on target virtual machine deployment information.

[0056] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0057] Generate initialized virtual machine deployment information, wherein the virtual machine deployment information is used to indicate that multiple virtual machines are deployed to multiple physical machines in a cloud computing environment;

[0058] Obtaining first physical machine resource information in a future time period when multiple physical machines have not deployed virtual machines, and virtual machine resource information in a future time period when multiple virtual machines have not been deployed to physical machines;

[0059] Prediction step: predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information;

[0060] Detecting environmental resource evaluation information of the cloud computing environment according to resource information of a second physical machine of the multiple physical machines in a future time period, wherein the environmental resource evaluation information represents at least one of a resource balance and energy consumption of the cloud computing environment;

[0061] Iteratively update the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information to obtain the target virtual machine deployment information;

[0062] Deploy multiple virtual machines to multiple physical machines based on target virtual machine deployment information.

[0063] The above-mentioned virtual machine deployment method, apparatus, computer equipment, computer-readable storage medium and computer program product take into account the impact of virtual machine deployment to physical machines on the environment in this application. Specifically, based on the first physical machine resource information of multiple physical machines in the future time period when virtual machines are not deployed, and the virtual machine resource information of multiple virtual machines in the future time period when multiple virtual machines are not deployed to physical machines, the second physical machine resource information of multiple physical machines in the future time period is predicted when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information, and based on the second physical machine resource information of multiple physical machines in the future time period, at least one environmental resource evaluation information of the resource balance and energy consumption of the cloud computing environment is detected to comprehensively consider the impact of environmental factors on the deployment process; finally, based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information, the virtual machine deployment information process is gradually optimized to obtain the target virtual machine deployment information, so as to achieve accurate deployment of multiple virtual machines to multiple physical machines according to the target virtual machine deployment information, so as to maximize the use of physical machine resources in an unstable cloud computing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0065] FIG1 is an application environment diagram of a virtual machine deployment method in one embodiment;

[0066] Figure 2 A schematic diagram of a flow chart of a virtual machine deployment method in an embodiment;

[0067] Figure 3 A schematic diagram of a network structure of a GRU (Gated Recurrent Unit) in one embodiment;

[0068] Figure 4 A flowchart of a virtual machine deployment method in a detailed embodiment;

[0069] Figure 5 A schematic diagram of the structure of a reinforcement learning model in one embodiment;

[0070] Figure 6 A schematic diagram of virtual machine deployment based on reinforcement learning in one embodiment;

[0071] Figure 7 It is a structural block diagram of a virtual machine deployment device in one embodiment;

[0072] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are used to explain the present application and are not used to limit the present application.

[0074] Virtualization technology, as the core foundation of cloud computing, provides the possibility of simulating multiple independent computer systems on a single physical host. The location of virtual machines in the data center is crucial because it directly affects the number of active hosts, which determines the energy consumption level of the data center. These virtual machines are able to run different operating systems and applications, which not only achieves efficient resource utilization, but also gives great flexibility in computing resource configuration and ensures rapid business response. In the cloud computing environment, virtual machine resource scheduling is a key technology to achieve more flexible data center resource integration. Through virtual machine resource scheduling, physical servers can be maintained, upgraded or reconfigured without affecting business continuity, thereby optimizing resource utilization.

[0075] Virtual machine deployment in cloud computing resource scheduling involves how to effectively allocate these virtual machines to physical servers. Its goal is to maximize the use of physical server resources while ensuring the performance and availability of virtual machines.

[0076] There are two main scenarios for virtual machine deployment: the first is initial deployment, when a new virtual machine is created, the system will select an appropriate host for the virtual machine deployment; the second is virtual machine deployment due to different reasons, the system needs to reallocate the host to the virtual machine. When dealing with virtual machine deployment, several key factors need to be considered, including system power consumption, service quality, and load balancing. Improper host selection will increase system power consumption, reduce service quality, uneven load distribution, and increase the frequency of repeated virtual machine deployment.

[0077] Therefore, the deployment of virtual machines is not only related to the rational allocation of resources, but also involves multiple aspects such as load balancing and performance optimization, which is crucial to improving the overall efficiency and reliability of cloud computing.

[0078] Through investigation of current research, it is found that the current virtual machine deployment strategy still has the following disadvantages:

[0079] 1. Virtual machine deployment decisions are basically made based on the current status of the virtual machine and host, ignoring the unstable nature of the cloud computing environment. The status of virtual machines and physical machines is not static; 2. When calculating the load value related to the physical machine during the virtual machine deployment strategy decision-making process, the impact of virtual machine deployment on the physical machine status is not taken into account; 3. The virtual machine deployment plan is decided through simple mathematical methods or heuristic algorithms, and the optimization effect for problems in the cloud computing environment is poor.

[0080] Therefore, providing an accurate virtual machine deployment method is crucial to improving the overall performance and efficiency of cloud computing systems.

[0081] The virtual machine deployment method provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers.

[0082] First, the user initiates a virtual machine deployment request in the virtual machine deployment interface of the terminal 102, and the server 104 randomly generates an initialized virtual machine deployment information based on the virtual machine deployment request, wherein the virtual machine deployment information is used to indicate that multiple virtual machines are deployed to multiple physical machines in the cloud computing environment; obtains the first physical machine resource information of multiple physical machines in a future time period when no virtual machines are deployed on the multiple physical machines, and the virtual machine resource information of multiple virtual machines in a future time period when no virtual machines are deployed on the physical machines; a prediction step: based on the first physical machine resource information and the virtual machine resource information, predicts the second physical machine resource information of the multiple physical machines in a future time period when multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information; based on the second physical machine resource information of the multiple physical machines in the future time period, detects the environmental resource evaluation information of the cloud computing environment, wherein the environmental resource evaluation information represents at least one of the resource balance and energy consumption of the cloud computing environment; based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information, iteratively updates the virtual machine deployment information to obtain the target virtual machine deployment information; and deploys the multiple virtual machines to the multiple physical machines according to the target virtual machine deployment information.

[0083] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0084] In an exemplary embodiment, Figure 2 As shown, a virtual machine deployment method is provided, and the method is applied to the server 104 in FIG. 1 as an example for description. In which:

[0085] S100: Generate initialized virtual machine deployment information.

[0086] The virtual machine deployment information is used to instruct to deploy multiple virtual machines to multiple physical machines in a cloud computing environment.

[0087] Specifically, the user initiates a virtual machine deployment request in the virtual machine deployment interface of the terminal. The server randomly generates a virtual machine deployment information based on the virtual machine deployment request. The virtual machine deployment information includes multiple deployment pairs. Each deployment pair consists of a physical machine and a virtual machine deployed to the physical machine. The virtual machine deployed to the physical machine may be 0 or not 0.

[0088] According to the randomly generated virtual machine deployment information, the deployment process is simulated to realize the deployment of multiple virtual machines to the physical machine corresponding to each virtual machine. Further, the physical machine corresponding to each virtual machine is determined by multiple deployment pairs in the virtual machine deployment information, and for each deployment pair, the virtual machine in the deployment pair is deployed to the physical machine in the deployment pair.

[0089] Furthermore, the virtual machine deployment information includes virtual machine deployment when creating a virtual machine (selecting a suitable physical machine for the newly created virtual machine to deploy), and virtual machine deployment during the virtual machine deployment process (selecting a suitable physical machine for the virtual machine to be migrated).

[0090] It should be noted that virtual machines do not necessarily need to be deployed on physical machines. The virtual machines in this application refer to virtual machines that need to be deployed.

[0091] S200, obtaining first physical machine resource information in a future time period when multiple physical machines have not deployed virtual machines, and virtual machine resource information in a future time period when multiple virtual machines have not been deployed to physical machines.

[0092] Specifically, the present application targets the unstable characteristics of the cloud computing environment, monitors n virtual machines and m physical machines on which virtual machines can be deployed in the cloud environment, and models the historical time series data of the virtual machines and physical machines, thereby predicting the resource information of the virtual machines and physical machines in future time periods. The resource information of the virtual machines and physical machines predicted in the future time periods at this time is the resource information when no virtual machines are deployed.

[0093] That is to say, historical virtual machine resource information of virtual machines and historical physical machine resource information of physical machines within a historical time period are obtained, and based on the historical virtual machine resource information, the first physical machine resource information in a future time period is predicted when multiple physical machines have not deployed virtual machines; and based on the historical physical machine resource information, the virtual machine resource information in a future time period is predicted when multiple virtual machines have not been deployed to physical machines.

[0094] Furthermore, the present application can predict resource information of virtual machines and physical machines in future time periods based on deep learning methods. For example, based on historical virtual machine resource information of virtual machines in historical time periods, the GRU model is used to predict resource information of the first physical machine in future time periods when multiple physical machines are not deployed with virtual machines, and the GRU model is used to predict resource information of virtual machines in future time periods when multiple virtual machines are not deployed to physical machines.

[0095] Among them, GRU is designed to solve the gradient vanishing and gradient exploding problems that may occur when RNN (Recurrent Neural Network) processes long sequences. It controls the flow of information by introducing two gates (reset gate and update gate). These two gates allow the network to remember important information and forget irrelevant information, so that it can perform better when processing long sequences. Furthermore, when processing sequence data, the internal mechanism of GRU allows it to weigh the dependence on historical information based on the update gate, while using the reset gate to filter and discard information that is no longer relevant. In this process, the sigmoid function plays a key role in determining which information should be retained. This design provides GRU with memory, enabling it to effectively capture the complex associations between long sequences. The network structure diagram of GRU is shown below. Figure 3 As shown, where x t is the current input information, h t-1 is the hidden state at the previous moment. The hidden state acts as the neural network memory, which contains information about the data that the node has seen before. t is the hidden state passed to the next moment, is a candidate hidden state, r t is the reset gate, z t It is the update door.

[0096] When using the GRU model to predict the virtual machine resource information of the virtual machine in the future time period, first, initialize the GRU model parameters, take the historical virtual machine resource information of the virtual machine in the historical time period as the input variable, and use the GRU network to learn the historical virtual machine resource information. The GRU network can capture the timing change rules in the data, thereby effectively predicting the first physical machine resource information in the future time period when multiple physical machines are not deployed with virtual machines. Similarly, when using the GRU model to predict the virtual machine resource information in the future time period when multiple virtual machines are not deployed to physical machines, the above steps are also used, and no further description is given here.

[0097] S300, predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information.

[0098] Specifically, the first physical machine resource information of all physical machines and the virtual machine resource information of all virtual machines deployed on each physical machine are resource information when no virtual machines are deployed in the future time period. For each physical machine, the second physical machine resource information of the physical machine is generated based on the first physical machine resource information of the physical machine and the virtual machine resource information of all virtual machines deployed on the physical machine. In practical applications, the second physical machine resource information is actually resource usage information.

[0099] S400: Detect environmental resource evaluation information of a cloud computing environment according to second physical machine resource information of multiple physical machines in a future time period.

[0100] The environmental resource assessment information represents at least one of the resource balance and energy consumption of the cloud computing environment.

[0101] Specifically, based on the resource information of the second physical machine of the multiple physical machines in the future time period, the environmental resource evaluation information of the cloud computing environment can be detected, and the environmental resource evaluation information includes at least one of the resource balance and energy consumption of the cloud computing environment. In other words, the environmental resource evaluation information can include the resource balance of the cloud computing environment, the environmental resource evaluation information can also include the energy consumption of the cloud computing environment, and the environmental resource evaluation information can also include the resource balance and energy consumption of the cloud computing environment.

[0102] By using at least one of the environmental resource evaluation information of resource balance and energy consumption of the cloud computing environment, the suitability of the current deployment based on the virtual machine deployment information is comprehensively and appropriately evaluated. In actual applications, there is more than one second physical machine resource information and more than one environmental resource evaluation information.

[0103] S500, iteratively updating the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information to obtain the target virtual machine deployment information.

[0104] Specifically, through the reinforcement learning model, the reinforcement learning algorithm is used to model various indicators in the cloud environment and make decisions on which physical machine each virtual machine is deployed to. In the reinforcement learning algorithm, deployment strategy reward information is generated based on environmental resource evaluation information, and based on the deployment strategy reward information, the virtual machine deployment information is iteratively updated to finally generate the target virtual machine deployment information.

[0105] In an exemplary embodiment, when the environmental resource assessment information represents the resource balance and energy consumption of the cloud computing environment, deployment strategy reward information is generated based on the environmental resource assessment information, including: weighting the resource balance and energy consumption to generate the deployment strategy reward information.

[0106] Specifically, in traditional virtual machine deployment strategies, the construction of deployment strategy reward information mainly focuses on single-dimensional indicators. However, in complex cloud computing environments, a single indicator often leads to energy consumption beyond the control range or unbalanced resource allocation and management. In order to make up for this deficiency, this application integrates the deployment environment resource balance with the deployment environment energy consumption information to generate deployment strategy reward information, which more finely represents the multi-dimensional resource status of the host.

[0107] That is to say, the resource balance and energy consumption in the cloud computing environment are weightedly fused to generate deployment strategy reward information. The weighted fusion method can be: obtaining a first weight coefficient of the resource balance and a second weight coefficient of the energy consumption, and updating the resource balance based on the first weight coefficient to obtain the weighted resource balance, and updating the energy consumption based on the second weight coefficient to obtain the weighted energy consumption.

[0108] Furthermore, the resource balance and energy consumption are weighted to generate the deployment strategy reward information expression including: .

[0109] in, is the first weight coefficient, BL is the resource balance degree, is the second weight coefficient, E is the energy consumption, the first weight coefficient and the second weight coefficient are determined according to the actual situation of each environment, and their sum is 1.

[0110] S600: Deploy multiple virtual machines onto multiple physical machines according to target virtual machine deployment information.

[0111] Specifically, multiple virtual machines are deployed to multiple physical machines according to the target virtual machine deployment information. Further, the target virtual machine deployment information also includes multiple deployment pairs, and multiple virtual machines are deployed to the physical machine corresponding to each virtual machine according to all the deployment pairs.

[0112] In the above-mentioned virtual machine deployment method, the impact of deploying virtual machines to physical machines on the environment is taken into consideration in this application. Specifically, based on the first physical machine resource information of multiple physical machines in the future time period when virtual machines are not deployed, and the virtual machine resource information of multiple virtual machines in the future time period when multiple virtual machines are not deployed to physical machines, the second physical machine resource information of multiple physical machines in the future time period is predicted when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information, and based on the second physical machine resource information of multiple physical machines in the future time period, at least one environmental resource evaluation information of the resource balance and energy consumption of the cloud computing environment is detected to comprehensively consider the impact of environmental factors on the deployment process; finally, based on the deployment strategy reward information corresponding to the virtual machine deployment information generated based on the environmental resource evaluation information, the virtual machine deployment information process is gradually optimized to obtain the target virtual machine deployment information, so as to accurately deploy multiple virtual machines to multiple physical machines according to the target virtual machine deployment information, so as to maximize the use of physical machine resources in an unstable cloud computing environment.

[0113] In an exemplary embodiment, based on the first physical machine resource information and the virtual machine resource information, predicting the second physical machine resource information of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information includes:

[0114] Obtain the deployment time step of the virtual machine in the virtual machine deployment information, and determine the future time period corresponding to the deployment time step; based on the first physical machine resource information and the virtual machine resource information at each time step, predict the second physical machine resource information of the multiple physical machines at each time step when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information; based on the second physical machine resource information of the multiple physical machines at each time step, predict the second physical machine resource information of the multiple physical machines in the future time period.

[0115] The future time period includes multiple time steps.

[0116] Specifically, the virtual machine deployment information is not only used to indicate that multiple virtual machines are deployed to multiple physical machines in the cloud computing environment, but also to indicate a specific deployment time step of the deployment, and to determine a future time period where the deployment time step is located, and the future time period is composed of multiple time steps. Further, the future time period where the deployment time step is located may be a future time period starting from the deployment time step, and the future time period where the deployment time step is located may also be a future time period obtained by combining a first time period before the deployment time step, the deployment time step, and a second time period after the deployment time step.

[0117] First, for each time step, obtain the first physical machine resource information of each time step in the future time period when multiple physical machines are not deployed with virtual machines, and the virtual machine resource information of each time step in the future time period when multiple virtual machines are not deployed to physical machines.

[0118] Secondly, based on the first physical machine resource information and virtual machine resource information at each time step, second physical machine resource information of multiple physical machines at each time step is predicted when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information.

[0119] Finally, based on the second physical machine resource information of the multiple physical machines at all time steps, the second physical machine resource information of the multiple physical machines in the future time period is predicted.

[0120] In an exemplary embodiment, based on the second physical machine resource information of the multiple physical machines at all time steps, the second physical machine resource information of the multiple physical machines in the future time period is predicted, including: taking the average of the second physical machine resource information of the multiple physical machines at all time steps as the second physical machine resource information of the multiple physical machines in the future time period.

[0121] In the above embodiment, by determining the future time period corresponding to the deployment time step, it is possible to accurately predict the second physical machine resource information of multiple physical machines at each time step when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information based on the first physical machine resource information and virtual machine resource information at each time step, thereby accurately predicting the second physical machine resource information of multiple physical machines in the future time period.

[0122] In an exemplary embodiment, the virtual machine resource information includes the effective use time of the virtual machine processor and the virtual machine memory occupancy; the physical machine resource information includes the effective use time of the physical machine processor and the physical machine memory occupancy, and the second physical machine resource information includes the processor usage rate and the memory usage rate;

[0123] like Figure 4 As shown, S300, including:

[0124] S320, predicting, based on the effective use time of the physical machine processor and the effective use time of the virtual machine processor, the processor usage rates of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information.

[0125] S340, predicting, based on the memory usage of the physical machine and the memory usage of the virtual machine, the memory usage of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information.

[0126] Among them, processor utilization is a measure of the amount of time the processor spends executing various tasks and processes on the computer. Memory utilization is a measure of the amount of time the memory spends executing various tasks and processes on the computer.

[0127] Specifically, at this time, the virtual machine deployment has been completed based on the initialized virtual machine deployment information, and all virtual machines have been deployed on the corresponding physical machines. The physical machine performance information at this time, that is, the processor utilization and memory utilization, can be constructed based on the virtual machine resource information of multiple virtual machines and the first physical machine resource information of multiple physical machines.

[0128] That is, based on the effective use time of the physical processors of all physical machines and the effective use time of the virtual machine processors of all virtual machines, the processor usage rates of multiple physical machines at this time in the future time period are predicted. At the same time, based on the physical memory usage of all physical machines and the virtual machine memory usage of all virtual machines, the memory usage of multiple physical machines at this time in the future time period is predicted.

[0129] Furthermore, when the virtual machine deployment information is subsequently updated, a physical machine performance matrix may be constructed based on the physical machine performance information in each cycle.

[0130] In the above embodiment, by predicting the processor usage and memory usage of multiple physical machines in future time periods when multiple virtual machines are deployed on multiple physical machines based on virtual machine deployment information, the performance of the physical machines can be accurately measured.

[0131] In an exemplary embodiment, the virtual machine resource information includes the effective use time of the virtual machine processor and the virtual machine memory occupancy; the physical machine resource information includes the effective use time of the physical machine processor and the physical machine memory occupancy. Generally, the processor is a CPU (Central Processing Unit).

[0132] The step of obtaining first physical machine resource information in a future time period when multiple physical machines are not deployed with virtual machines, and virtual machine resource information in a future time period when multiple virtual machines are not deployed to physical machines may also include:

[0133] First, obtain the following data in the historical time period: the i-th virtual machine has p consecutive equal time periods [t0, t1, …, t p ] can be expressed as , the jth physical machine in p consecutive time periods [t0, t1, …, t p ] can be expressed as , the i-th virtual machine at p consecutive time points [time0, time1, …, time p The memory usage of ] can be expressed as , the jth physical machine at p consecutive time points [time0, time1, …, time p The memory usage of ] can be expressed as .

[0134] Secondly, based on the deep learning model GRU, the continuous time period [t0, t1, ..., t p ] to model the historical virtual machine CPU effective usage time within the future continuous time period [t p+1 , t p+2 ,…,t p+q ] to predict the effective time of the CPU in the continuous time period [t0, t1, ..., t p ] Historical virtual machine CPU effective usage time As input variables, through this learning process, the GRU network generates the effective CPU usage time of the future time period, and obtains .

[0135] Similarly, the historical virtual machine memory usage m i , historical physical machine cpu effective usage time cpu j 、Historical physical machine memory usage mem j They are input as initial values ​​into the GRU model to obtain the predicted virtual machine memory usage m i '、Physical machine CPU effective usage time cpu j 'And the physical machine memory usage mem j '.

[0136] In an exemplary embodiment, predicting the processor usage rates of multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on virtual machine deployment information according to the effective use time of the physical machine processor and the effective use time of the virtual machine processor includes:

[0137] For any target physical machine among the multiple physical machines, obtain the total execution time of the processor of the target physical machine;

[0138] Based on the total processor execution time, the effective usage time of the physical machine processor of the target physical machine, and the effective usage time of the virtual machine processor of each virtual machine deployed on the target physical machine, the processor usage rate of the target physical machine in the future time period is predicted when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information.

[0139] Specifically, each physical machine may or may not have a virtual machine deployed on it. Regardless of whether a virtual machine is deployed on the physical machine, the physical machine processor usage rate can be generated based on the effective use time of the physical machine processor and the effective use time of the virtual machine processors of all virtual machines deployed on each physical machine.

[0140] Furthermore, when generating the physical machine processor usage rate based on the effective usage time of the physical machine processor and the effective usage time of the virtual machine processors of all virtual machines deployed on each physical machine, it is also necessary to obtain the total processor execution time of the target physical machine, that is, to combine the total processor execution time, the effective usage time of the physical machine processor of the target physical machine, and the effective usage time of the virtual machine processor of each virtual machine deployed on the target physical machine, to predict the processor usage rate of the target physical machine in the future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information.

[0141] At this time, the expression of the processor usage of the jth physical machine in the tth time step is as follows:

[0142]

[0143] Among them, k ij Indicates whether the i-th virtual machine is deployed on the j-th physical machine, k ij =1 means the i-th virtual machine is deployed on the j-th physical machine, k ij =0 means that the i-th virtual machine is not deployed on the j-th physical machine; represents the effective usage time of the virtual machine processor of the i-th virtual machine in the predicted t-th time step, It indicates the effective usage time of the physical processor of the jth physical machine in the predicted tth time step; Indicates the total execution time of the physical machine processor. The total execution time of the physical machine processor must be greater than or equal to the effective use time of the physical machine processor; It represents the processor usage of the jth physical machine at the tth time step. This calculation method takes the deployment of virtual machines into consideration, and the obtained physical machine processor usage can better reflect the actual situation of the physical machine.

[0144] Based on the memory usage of the physical machine and the memory usage of the virtual machine, the memory usage of multiple physical machines in the future time period is predicted when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information, including:

[0145] For any target physical machine among multiple physical machines, the total memory execution time of the target physical machine is obtained; based on the total memory execution time, the physical machine memory occupancy of the target physical machine, and the virtual machine memory occupancy of each virtual machine deployed on the target physical machine, the memory usage of the target physical machine in the future time period is predicted when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information.

[0146] Similarly, regardless of whether a virtual machine is deployed on the physical machine, the memory usage of the physical machine can be generated according to the memory usage of the physical machine and the virtual machine memory usage of all virtual machines deployed on each physical machine.

[0147] Furthermore, when generating the memory utilization rate of the physical machine based on the memory occupancy of the physical machine and the virtual machine memory occupancy of all virtual machines deployed on each physical machine, it is also necessary to obtain the total memory execution time of the target physical machine, that is, to predict the memory utilization rate of the target physical machine in the future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information, based on the total memory execution time, the physical machine memory occupancy of the target physical machine, and the virtual machine memory occupancy of each virtual machine deployed on the target physical machine.

[0148] At this time, the expression of the memory usage of the jth physical machine in the tth time step is as follows:

[0149]

[0150] In the above formula, represents the predicted virtual machine memory usage of the i-th virtual machine in the t-th time step, It represents the predicted physical machine memory usage of the j-th physical machine in the t-th time step; Indicates the total memory execution time of the physical machine. The total memory execution time of the physical machine refers to the effective memory execution time of the physical machine. The total memory execution time of the physical machine must be greater than or equal to the effective memory usage time of the physical machine. Represents the memory usage of the jth physical machine in the tth time step.

[0151] In an exemplary embodiment, in order to reduce the interference caused by a single time step to the detection and achieve more accurate detection of environmental resource assessment information, the present application can also obtain the processor usage and memory usage of each physical machine in multiple time steps in the future time period, and based on the processor usage of each physical machine in all time steps, generate the average processor usage of the physical machine, and use the average processor usage as the processor usage, and based on the memory usage of each physical machine in all time steps, generate the average memory usage of the physical machine, and use the average memory usage as the memory usage of the physical machine.

[0152] In this embodiment, after the deployment is completed, the processor usage of the physical machine can be accurately generated based on the effective usage time of the physical machine processor and the effective usage time of the virtual machine processors of all virtual machines deployed on each physical machine. At the same time, the memory usage of the physical machine can also be accurately generated based on the memory usage of the physical machine and the virtual machine memory usage of all virtual machines deployed on each physical machine.

[0153] In an exemplary embodiment, the environmental resource evaluation information represents the resource balance and energy consumption of the cloud computing environment; and detecting the environmental resource evaluation information of the cloud computing environment according to the second physical machine resource information of the plurality of physical machines in the future time period includes:

[0154] Obtain the number of physical machines of multiple physical machines, the full-load energy consumption of the physical machines when the multiple physical machines are in a fully loaded state, and the basic energy consumption of the physical machines when the multiple physical machines are in an idle state; detect the resource balance of the cloud computing environment based on the processor utilization rate, memory utilization rate and number of physical machines of the multiple physical machines in a future time period; detect the energy consumption of the cloud computing environment based on the processor utilization rate of the multiple physical machines in a future time period, the full-load energy consumption of the physical machines when the multiple physical machines are in a fully loaded state, and the basic energy consumption of the physical machines when the multiple physical machines are in an idle state.

[0155] The physical machine being in a fully loaded state means that no new virtual machines can be deployed on the physical machine. The physical machine being in an idle state means that no virtual machines are deployed on the physical machine.

[0156] Specifically, in order to optimize the deployment of virtual machines to achieve a more balanced allocation of resources, the present invention adopts a resource balance evaluation method based on relative ratios. When the ratio of the processor utilization rate to the memory utilization rate of a physical machine is closer to the ideal value of 1, it means that the resource utilization of the physical machine is more balanced, thereby achieving effective and efficient utilization of resources. Therefore, the ratio of the processor utilization rate to the memory utilization rate of each physical machine is obtained, and based on the utilization rate ratio of all physical machines, the resource balance in the cloud computing environment is detected. In addition, detecting the resource balance in the cloud computing environment also requires the participation of the number of physical machines of multiple physical machines.

[0157] Furthermore, the expression of resource balance in cloud computing environment is as follows:

[0158]

[0159] Where m is the number of physical machines, cv j is the processor usage of the jth physical machine, mv j is the memory usage of the jth physical machine.

[0160] In addition, usually, the energy consumption of the cloud computing environment is often positively correlated with the energy consumption of the physical machine in the cloud computing environment, and the energy consumption of the physical machine is directly positively correlated with the processor usage of the physical machine. Therefore, the energy consumption of the cloud computing environment can also be detected based on the processor usage of the physical machine. The specific steps are as follows:

[0161] The full-load energy consumption of multiple physical machines in full-load state and the basic energy consumption of multiple physical machines in idle state are obtained, and the energy consumption of the cloud computing environment is detected according to the processor usage rates of the multiple physical machines in the future time period, the full-load energy consumption of the multiple physical machines in full-load state and the basic energy consumption of the multiple physical machines in idle state.

[0162] Among them, according to the processor utilization rate of multiple physical machines in the future time period, the full load energy consumption of multiple physical machines in full load state and the basic energy consumption of multiple physical machines in idle state, the energy consumption of the cloud computing environment is detected, including: for any physical machine, based on the physical machine processor utilization rate of the physical machine in the future time period, the full load energy consumption of the physical machine in full load state and the basic energy consumption of the physical machine in idle state, the physical machine energy consumption of the physical machine is generated; and based on the physical machine energy consumption of all physical machines, the energy consumption in the cloud computing environment is detected.

[0163] Furthermore, the energy consumption of the cloud computing environment is expressed as follows:

[0164]

[0165] Among them, ec j is the physical machine energy consumption of each physical machine, m is the number of physical machines, and E is the energy consumption of the cloud computing environment.

[0166] The steps for generating the physical machine energy consumption of each physical machine are as follows:

[0167] According to the basic energy consumption and full-load energy consumption of each physical machine, the energy consumption ratio of each physical machine is detected, and based on the full-load energy consumption, energy consumption ratio and memory usage of each physical machine, the physical machine energy consumption of each physical machine is generated.

[0168] The energy consumption of each physical machine under full load is mainly calculated by multiplying power by time. The specific steps include:

[0169] (1) Power determination: First, you need to know the power (P) of the physical machine when it is fully loaded. This can usually be found in the technical specifications of the physical machine or obtained through actual measurement.

[0170] (2) Time factor: Next, determine the running time (t), which is the total time the physical machine runs under full load.

[0171] (3) Energy consumption calculation: The full-load energy consumption of a physical machine is calculated using the formula E = P × t, expressed in watt-hours (Wh) or kilowatt-hours (kWh).

[0172] Idle physical machine energy consumption refers to the basic energy consumption of each physical machine, that is, the physical machine will also consume a certain amount of energy when it is idle or in standby state, which is called basic energy consumption.

[0173] According to the basic energy consumption of each physical machine in an idle state and the full-load energy consumption in a fully loaded state, the percentage of the energy consumption in the idle state to the energy consumption in the fully loaded state is detected, that is, the energy consumption ratio of each physical machine is detected.

[0174] Finally, the energy consumption ratio is detected according to the basic energy consumption and full-load energy consumption of each physical machine, and the physical machine energy consumption of each physical machine is generated based on the full-load energy consumption, energy consumption ratio and memory usage.

[0175] Furthermore, the expression for generating the energy consumption of the physical machine is as follows:

[0176]

[0177] Among them, cv j is the memory usage of the jth physical machine, is the full load energy consumption of the jth physical machine under full load, is the energy consumption ratio of the jth physical machine, is the physical machine energy consumption of the jth physical machine.

[0178] In the above embodiment, by obtaining the number of physical machines of multiple physical machines, based on the processor utilization rate, memory utilization rate and the number of physical machines of the multiple physical machines in the future time period, the resource balance of the cloud computing environment can be accurately detected; at the same time, by obtaining the full load energy consumption of the physical machine in a full load state and the basic energy consumption of the physical machine in an idle state, the energy consumption ratio between the full load energy consumption of the physical machine in a full load state and the basic energy consumption of the physical machine in an idle state can be accurately detected, and then based on the full load energy consumption, the energy consumption ratio and the memory utilization rate, the energy consumption of the physical machine can be accurately and efficiently generated.

[0179] In an exemplary embodiment, based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information, the virtual machine deployment information is iteratively updated to obtain the target virtual machine deployment information, including:

[0180] Based on the environmental resource evaluation information, deployment strategy reward information corresponding to the virtual machine deployment information is generated; based on the deployment strategy reward information, the first physical machine resource information and the virtual machine resource information are updated, and based on the updated first physical machine resource information and the virtual machine resource information, the virtual machine deployment information is updated; return to the prediction step until the deployment strategy reward information converges or the number of iterations reaches a preset number threshold, and according to the deployment strategy reward information generated during the multiple rounds of iterations, the target virtual machine deployment information is selected from the virtual machine deployment information updated during the multiple rounds of iterations.

[0181] Specifically, based on the environmental resource assessment information, deployment strategy reward information corresponding to the virtual machine deployment information is generated, based on the deployment strategy reward information, the first physical machine resource information and the virtual machine resource information are updated, and based on the updated first physical machine resource information and the virtual machine resource information, the virtual machine deployment information is updated, and finally, the deployment step is returned to be executed until the deployment strategy reward information converges or the number of iterations reaches a preset threshold, and finally, a virtual machine deployment information is selected from the virtual machine deployment information updated in all rounds of iterations as the target virtual machine deployment information.

[0182] Generally, since the deployment strategy reward information is determined by resource balance and energy consumption, the lower the energy consumption and balance, the better the overall environment, and the lower the balance value, the more balanced it is. Therefore, the deployment strategy reward information can be used to filter the target deployment strategy from all virtual machine deployment information. For example, the virtual machine deployment information in the round with the smallest deployment strategy reward value corresponding to the deployment strategy reward information is selected as the target virtual machine deployment information.

[0183] In an exemplary embodiment, the iterative update step of the virtual machine deployment information of the present application is actually implemented through a reinforcement learning algorithm.

[0184] Among them, the reinforcement learning algorithm has the ability to optimize parameters through continuous learning to achieve the optimization of the objective function. Considering that the virtual machine deployment method is essentially a process of pursuing the best solution, its internal logic is consistent with the core concept of reinforcement learning. In addition, compared with traditional machine learning algorithms, reinforcement learning has shown significant advantages in dealing with decision-making problems in complex environments, especially when dealing with environments with large state spaces and frequent dynamic changes. The algorithm gradually learns and optimizes the deployment strategy through direct interaction with the environment, without relying on prior knowledge, and has the ability to achieve autonomous exploration and continuous optimization in complex situations. In layman's terms, reinforcement learning is a branch of machine learning. It interacts with the environment through the agent and guides learning based on the rewards given by the environment. The goal is to enable the agent to learn to take the best action in a specific environment to obtain the maximum reward. Reinforcement learning is based on trial and error learning. The agent selects an action to execute based on the current state of the environment, and the environment gives a new state and reward after receiving the action. The agent updates its strategy based on this reward, that is, the rules for selecting different actions in different states in order to obtain greater rewards in the future.

[0185] Therefore, this application uses a reinforcement learning algorithm to generate the final target virtual machine deployment information, and the virtual machine deployment method based on reinforcement learning is modeled as follows:

[0186] (1) Input: The input data includes a list of physical machines in the environment where virtual machines can be deployed and a list of virtual machines to be allocated.

[0187] (2) State space: The state space of the algorithm is composed of the second physical machine resource information obtained based on the first physical machine resource information and the virtual machine resource information.

[0188] (3) Action space: Each virtual machine selects a physical machine for deployment, which can be represented as an n-row and m-column matrix. The i-th row and j-th column indicate whether the i-th virtual machine is deployed on the j-th physical machine. If yes, it is 1, and if not, it is 0.

[0189] (4) Reward function: Generates deployment strategy reward information r based on environmental resource evaluation information.

[0190] Based on this strategy modeling, it can be seen that, first, the first physical machine resource information in the future time period when multiple physical machines have not deployed virtual machines, and the virtual machine resource information in the future time period when multiple virtual machines have not been deployed to physical machines are obtained. Based on this information, the second physical machine resource information can be generated, and the second physical machine resource information is regarded as the initial state space. At this time, the initial action space is the initialized virtual machine deployment information. Based on the initial state space and the initial action space, the deployment strategy reward information and the new state space are generated. Since the state space is generated by the first physical machine resource information and the virtual machine resource information, that is, new virtual machine deployment information is generated, and the first physical machine resource information and the virtual machine resource information are updated. The reason for generating a new state space is that multiple virtual machines have been deployed to the physical machine corresponding to each virtual machine. As time goes by, the resource information of the physical machine and the virtual machine will gradually change. Therefore, the second physical machine resource information of all physical machines calculated based on all resource information will be updated.

[0191] In an exemplary embodiment, the reinforcement learning model structure diagram is as follows: Figure 5 As shown, according to the deployment process between each virtual machine and physical machine, the current action space is generated to act on the cloud computing environment, and the deployment strategy reward information r and the new state space are output. Based on the deployment strategy reward information r and the new state space, the deployment process of the virtual machine is acted on to generate a new action space, and this cycle is repeated.

[0192] In an exemplary embodiment, a virtual machine deployment diagram based on reinforcement learning is shown in FIG. Figure 6 As shown, the intelligent agent executes the current action and deploys the virtual machines to be deployed in all virtual machines to the physical machine. At this time, it monitors environmental changes, obtains environmental resource evaluation information, and generates deployment strategy reward information based on the environmental resource evaluation information. The deployment strategy reward information is then applied to the intelligent agent to generate a new deployment action.

[0193] Specifically, based on the deployment strategy reward information, the first physical machine resource information and the virtual machine resource information are updated, and based on all the updated first physical machine resource information, the second physical machine resource information is updated, and then based on the reward and the updated second physical machine resource information, the virtual machine deployment information is updated to obtain a new action space in a new state space. The new action space in a new state space is not randomly generated, but the action with the largest state-action value function Q is taken from all possible actions in the current state.

[0194] Then, based on the new virtual machine deployment information, multiple virtual machines are deployed again to the physical machines corresponding to each virtual machine, and new deployment strategy reward information and deployment resource evaluation information of all physical machines are generated, that is, the state space is updated again, and the above steps are repeated until the deployment strategy reward information converges or the number of iterations reaches a preset threshold, the virtual machine deployment information updated in each round of iteration is summarized, and according to the deployment strategy reward information updated in multiple rounds of iterations, the target virtual machine deployment information is selected from the virtual machine deployment information updated in multiple rounds of iterations.

[0195] In an exemplary embodiment, the virtual machine deployment method based on reinforcement learning also includes:

[0196] (1) Take the list of virtual machines to be allocated and the list of available physical machines as input;

[0197] (2) The agent obtains the current deployment resource evaluation information based on all resource information, lists it and uses it as the state space;

[0198] (3) Initialize the Q-table, whose size matches the state space and action space, and all Q values ​​are initialized to random values;

[0199] (4) At each time step, the agent makes action decisions based on the current state and generates the current action space;

[0200] (5) Calculate the deployment strategy reward information r based on the current action decision;

[0201] (6) Update the Q value based on the current state. The expression for updating the Q value is as follows:

[0202]

[0203] Among them, s is the current state, a is the current action, r is the observed deployment strategy reward information, s' is the new state entered after taking the current action a, a' is the action with the highest Q value in the new state, alpha is a parameter representing the learning rate, which is used to control the update step size, and gamma is a parameter representing the future reward value of the agent, that is, the discount factor, with a value range of [0, 1], which is used to balance the immediate reward and future rewards.

[0204] (7) Repeat steps (4) to (6) until the deployment strategy reward information r converges or the maximum number of iterations is reached.

[0205] In a detailed embodiment, the present application models the time series data of virtual machines and physical machines based on the GRU model in view of the unstable characteristics of the cloud computing environment, and predicts the load data of virtual machines and physical machines in the future time period based on the deep learning method; then, based on the predicted value and the situation of virtual machines deployed to different physical machines, the performance calculation method of the physical machine after the virtual machine deployment is given, and the physical machine resource utilization, resource balance, and total power consumption of the cloud environment are calculated based on the calculated physical machine performance data; finally, a virtual machine deployment strategy is made based on the reinforcement learning model. The details are as follows:

[0206] S1. Obtain the historical virtual machine CPU effective usage time and the historical virtual machine memory usage in the future time period when the multiple virtual machines are not deployed to the physical machines within the historical time period, and predict the virtual machine CPU effective usage time and the virtual machine memory usage in the future time period when the multiple virtual machines are not deployed to the physical machines based on the historical virtual machine CPU effective usage time and the historical virtual machine memory usage. Similarly, predict the physical machine CPU effective usage time and the physical machine memory usage in the future time period when multiple physical machines are not deployed with virtual machines.

[0207] S2. Randomly generate initialized virtual machine deployment information, and simulate deploying multiple virtual machines to the physical machine corresponding to each virtual machine according to the virtual machine deployment information.

[0208] S3. In the tth time step, for any target physical machine among the multiple physical machines, obtain the total processor execution time and the total memory execution time of the target physical machine, and predict the processor usage rate of the target physical machine in the tth time step when multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information based on the total processor execution time, the effective usage time of the physical machine processor of the target physical machine, and the effective usage time of the virtual machine processor of each virtual machine deployed on the target physical machine; predict the memory usage rate of the target physical machine in the tth time step when multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information based on the total memory execution time, the physical machine memory occupancy of the target physical machine, and the virtual machine memory occupancy of each virtual machine deployed on the target physical machine.

[0209] S4. The average of the CPU utilization rates of all physical machines in multiple time steps is used as the CPU utilization rate in the future time period, and the average of the memory utilization rates of all physical machines in multiple time steps is used as the memory utilization rate in the future time period. Based on the processor utilization rates, memory utilization rates and number of physical machines of multiple physical machines in the future time period, the resource balance of the cloud computing environment in the future time period is detected. For each physical machine, in the future time period, the full load energy consumption of the physical machine in a full load state and the basic energy consumption of the physical machine in an idle state are obtained, and the energy consumption ratio is detected based on the basic energy consumption and the full load energy consumption, and the physical machine energy consumption is generated based on the full load energy consumption, the energy consumption ratio and the memory utilization rate, and the energy consumption of the cloud computing environment in the future time period of the cloud computing environment is detected based on the physical machine energy consumption of all physical machines.

[0210] S5. Perform weighted processing on resource balance and energy consumption to generate deployment strategy reward information.

[0211] S6. Based on the deployment strategy reward information, update the first physical machine resource information and the virtual machine resource information, and based on the updated first physical machine resource information and the virtual machine resource information, update the virtual machine deployment information; return to the prediction step until the deployment strategy reward information converges or the number of iterations reaches a preset threshold, and select the target virtual machine deployment information from the virtual machine deployment information updated during the multiple rounds of iterations according to the deployment strategy reward information generated during the multiple rounds of iterations.

[0212] S7. Deploy multiple virtual machines onto multiple physical machines according to the target virtual machine deployment information.

[0213] Based on the above steps, in a cloud computing environment, since the performance of physical machines and virtual machines may be affected by a variety of factors (such as hardware aging, load fluctuations, network conditions, etc.), accurate performance prediction is crucial for optimizing virtual machine deployment, resource scheduling, etc. This application uses the GRU model to predict the performance of virtual machines and physical machines in future time periods. Through this virtual machine deployment algorithm optimization system based on performance prediction, it can better cope with the instability of the cloud computing environment and improve the reliability and performance of the system. Secondly, this application considers the impact of virtual machine deployment on physical machine performance, simulates virtual machine deployment to obtain dynamic physical machine performance data, and uses it as a basis to calculate the resource balance and total energy consumption of the environment, which has a significant optimization and enhancement effect on the virtual machine deployment algorithm. Finally, this application uses a reinforcement learning algorithm to make virtual machine deployment decisions. The reinforcement learning algorithm allows the system to learn from the interaction with the environment, and gradually optimizes its decision-making process by trying different behaviors (in this case, virtual machine deployment strategies) and observing the results (i.e., resource balance, environmental energy consumption). This has a significant optimization and enhancement effect on the virtual machine deployment algorithm, which helps to improve the performance, stability and energy efficiency of the cloud environment.

[0214] This application may be applied in the following scenarios in the future: 1. Virtual machine deployment when creating a virtual machine (selecting a suitable physical machine for the virtual machine to deploy); 2. Virtual machine deployment during the virtual machine deployment process (selecting a suitable physical machine for the virtual machine). Its possible value includes: This application has a significant optimization and enhancement effect on the virtual machine deployment algorithm, which helps to improve the performance, stability and energy efficiency of the cloud environment.

[0215] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0216] Based on the same inventive concept, the embodiment of the present application also provides a virtual machine deployment device for implementing the virtual machine deployment method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more virtual machine deployment device embodiments provided below can refer to the limitations of the virtual machine deployment method above, and will not be repeated here.

[0217] In an exemplary embodiment, Figure 7 As shown, a virtual machine deployment device is provided, including: a deployment information generation module 100, an initial resource information acquisition module 200, a prediction module 300, an environmental resource evaluation module 400, an iterative processing module 500 and a deployment module 600, wherein:

[0218] A deployment information generation module 100 is used to generate initialized virtual machine deployment information, wherein the virtual machine deployment information is used to indicate that multiple virtual machines are deployed to multiple physical machines in a cloud computing environment;

[0219] An initial resource information acquisition module 200 is used to acquire resource information of a first physical machine in a future time period when multiple physical machines have not deployed virtual machines, and resource information of virtual machines in a future time period when multiple virtual machines have not been deployed to physical machines;

[0220] Prediction module 300, used for prediction step: predicting second physical machine resource information of multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on virtual machine deployment information, based on the first physical machine resource information and the virtual machine resource information;

[0221] An environmental resource evaluation module 400 is used to detect environmental resource evaluation information of a cloud computing environment based on resource information of a second physical machine of multiple physical machines in a future time period, wherein the environmental resource evaluation information represents at least one of a resource balance and energy consumption of the cloud computing environment;

[0222] Iterative processing module 500, used to iteratively update the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information, to obtain the target virtual machine deployment information;

[0223] The deployment module 600 is used to deploy multiple virtual machines to multiple physical machines according to the target virtual machine deployment information.

[0224] In one embodiment, the prediction module 300 is also used to obtain the deployment time step of the virtual machine in the virtual machine deployment information, and determine the future time period corresponding to the deployment time step, wherein the future time period includes multiple time steps; based on the first physical machine resource information and virtual machine resource information of each time step, predict the second physical machine resource information of the multiple physical machines at each time step when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information; based on the second physical machine resource information of the multiple physical machines at each time step, predict the second physical machine resource information of the multiple physical machines in the future time period.

[0225] In one embodiment, the virtual machine resource information includes the effective usage time of the virtual machine processor and the memory usage of the virtual machine; the physical machine resource information includes the effective usage time of the physical machine processor and the memory usage of the physical machine, and the second physical machine resource information includes the processor usage rate and the memory usage rate; the prediction module 300 is also used to predict the processor usage rate of multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information based on the effective usage time of the physical machine processor and the effective usage time of the virtual machine processor; and predict the memory usage of multiple physical machines in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information based on the physical machine memory usage and the virtual machine memory usage.

[0226] In one embodiment, the prediction module 300 is also used to obtain the total processor execution time of any target physical machine among multiple physical machines; predict the processor usage rate of the target physical machine in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information based on the total processor execution time, the effective usage time of the physical machine processor of the target physical machine, and the effective usage time of the virtual machine processor of each virtual machine deployed on the target physical machine; the prediction module 300 is also used to obtain the total memory execution time of the target physical machine for any target physical machine among multiple physical machines; predict the memory usage rate of the target physical machine in a future time period when multiple virtual machines are deployed on multiple physical machines based on the virtual machine deployment information based on the total memory execution time, the physical machine memory occupancy of the target physical machine, and the virtual machine memory occupancy of each virtual machine deployed on the target physical machine.

[0227] In one of the embodiments, the environmental resource assessment information represents the resource balance and energy consumption of the cloud computing environment. The environmental resource assessment module 400 is further used to obtain the number of physical machines of multiple physical machines, the full load energy consumption of the physical machines when the multiple physical machines are in a fully loaded state, and the basic energy consumption of the physical machines when the multiple physical machines are in an idle state; based on the processor utilization rate, memory utilization rate and number of physical machines of the multiple physical machines in a future time period, the resource balance of the cloud computing environment is detected; based on the processor utilization rate of the multiple physical machines in a future time period, the full load energy consumption of the physical machines when the multiple physical machines are in a fully loaded state, and the basic energy consumption of the physical machines when the multiple physical machines are in an idle state, the energy consumption of the cloud computing environment is detected.

[0228] In one embodiment, the iterative processing module 500 is also used to generate deployment strategy reward information corresponding to the virtual machine deployment information based on the environmental resource evaluation information; based on the deployment strategy reward information, update the first physical machine resource information and the virtual machine resource information, and based on the updated first physical machine resource information and the virtual machine resource information, update the virtual machine deployment information; return to the prediction step until the deployment strategy reward information converges or the number of iterations reaches a preset threshold, and select the target virtual machine deployment information from the virtual machine deployment information updated during the multiple rounds of iterations based on the deployment strategy reward information generated during the multiple rounds of iterations.

[0229] Each module in the above virtual machine deployment device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0230] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the physical machine resource information of the physical machine and the virtual machine resource information data of all virtual machines. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a virtual machine deployment method is implemented.

[0231] Those skilled in the art will understand that Figure 8 The structure shown in the figure is a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0232] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0233] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0234] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0235] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0236] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0237] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A virtual machine deployment method, characterized in that: The method comprises: Generate initialized virtual machine deployment information, wherein the virtual machine deployment information is used to indicate that multiple virtual machines are deployed to multiple physical machines in a cloud computing environment; Acquire resource information of a first physical machine in a future time period when the multiple physical machines have no virtual machines deployed thereon, and resource information of a virtual machine in a future time period when the multiple virtual machines have not been deployed thereon; Prediction step: predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information; Detecting environmental resource evaluation information of the cloud computing environment according to the second physical machine resource information of the multiple physical machines in the future time period, wherein the environmental resource evaluation information represents at least one of resource balance and energy consumption of the cloud computing environment; Iteratively updating the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information to obtain target virtual machine deployment information; The multiple virtual machines are deployed onto the multiple physical machines according to the target virtual machine deployment information.

2. The method according to claim 1, characterized in that The predicting, according to the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information includes: Obtaining a deployment time step for deploying a virtual machine in the virtual machine deployment information, and determining a future time period corresponding to the deployment time step, wherein the future time period includes multiple time steps; Based on the first physical machine resource information and the virtual machine resource information at each time step, predicting the second physical machine resource information of the multiple physical machines at each time step when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information; Based on the second physical machine resource information of the multiple physical machines at each of the time steps, the second physical machine resource information of the multiple physical machines in the future time period is predicted.

3. The method according to claim 1, characterized in that The virtual machine resource information includes the effective use time of the virtual machine processor and the memory usage of the virtual machine; the first physical machine resource information includes the effective use time of the physical machine processor and the memory usage of the physical machine, and the second physical machine resource information includes the processor usage rate and the memory usage rate; The predicting, according to the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information includes: According to the effective use time of the physical machine processor and the effective use time of the virtual machine processor, predicting the processor usage rates of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information; According to the memory occupancy of the physical machine and the memory occupancy of the virtual machine, the memory occupancy of the multiple physical machines in a future time period is predicted when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information.

4. The method according to claim 3, characterized in that The predicting, based on the effective use time of the physical machine processor and the effective use time of the virtual machine processor, the processor usage rates of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information includes: For any target physical machine among the multiple physical machines, obtaining the total execution time of the processor of the target physical machine; According to the total execution time of the processor, the effective use time of the physical machine processor of the target physical machine, and the effective use time of the virtual machine processor of each virtual machine deployed on the target physical machine, predicting the processor usage rate of the target physical machine in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information; The predicting, according to the memory occupancy of the physical machine and the memory occupancy of the virtual machine, the memory occupancy of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information, includes: For any target physical machine among the multiple physical machines, obtaining the total memory execution time of the target physical machine; Based on the total memory execution time, the physical machine memory occupancy of the target physical machine, and the virtual machine memory occupancy of each virtual machine deployed on the target physical machine, predict the memory usage of the target physical machine in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information.

5. The method according to claim 3, characterized in that: The environmental resource evaluation information represents the resource balance and energy consumption of the cloud computing environment, and detecting the environmental resource evaluation information of the cloud computing environment according to the second physical machine resource information of the multiple physical machines in the future time period includes: Obtaining the number of physical machines of the multiple physical machines, the full-load energy consumption of the multiple physical machines when they are in a fully loaded state, and the basic energy consumption of the multiple physical machines when they are in an idle state; Detecting a resource balance of the cloud computing environment based on processor usage, memory usage, and the number of physical machines of the multiple physical machines in a future time period; The energy consumption of the cloud computing environment is detected according to the processor usage rates of the multiple physical machines in a future time period, the full load energy consumption of the multiple physical machines in a fully loaded state, and the basic energy consumption of the multiple physical machines in an idle state.

6. The method according to claim 1, characterized in that The deployment strategy reward information corresponding to the virtual machine deployment information generated based on the environmental resource evaluation information, iteratively updating the virtual machine deployment information to obtain the target virtual machine deployment information, includes: Based on the environmental resource evaluation information, generating deployment strategy reward information corresponding to the virtual machine deployment information; Based on the deployment strategy reward information, updating the first physical machine resource information and the virtual machine resource information, and based on the updated first physical machine resource information and the virtual machine resource information, updating the virtual machine deployment information; Return to the prediction step until the deployment strategy reward information converges or the number of iterations reaches a preset threshold, and select the target virtual machine deployment information from the virtual machine deployment information updated during the multiple rounds of iterations based on the deployment strategy reward information generated during the multiple rounds of iterations.

7. A virtual machine deployment device, characterized in that: The device comprises: A deployment information generation module, used to generate initialized virtual machine deployment information, wherein the virtual machine deployment information is used to indicate that multiple virtual machines are deployed to multiple physical machines in a cloud computing environment; An initial resource information acquisition module, used to acquire resource information of a first physical machine in a future time period when the multiple physical machines have not deployed virtual machines, and resource information of virtual machines in a future time period when the multiple virtual machines have not been deployed to the physical machines; A prediction module, configured to predict the following steps: predicting, based on the first physical machine resource information and the virtual machine resource information, second physical machine resource information of the multiple physical machines in a future time period when the multiple virtual machines are deployed on the multiple physical machines based on the virtual machine deployment information; an environmental resource evaluation module, configured to detect environmental resource evaluation information of the cloud computing environment according to resource information of a second physical machine of the multiple physical machines in a future time period, wherein the environmental resource evaluation information represents at least one of a resource balance and energy consumption of the cloud computing environment; An iterative processing module, configured to iteratively update the virtual machine deployment information based on the deployment strategy reward information corresponding to the virtual machine deployment information generated by the environmental resource evaluation information, to obtain target virtual machine deployment information; A deployment module is used to deploy the multiple virtual machines to the multiple physical machines according to the target virtual machine deployment information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images