Virtual machine deployment method and device and computer equipment

Abstract

The invention relates to a virtual machine deployment method and device and computer equipment, and the method comprises the steps: receiving resource request information sent by a client, obtaining the resource use information of a resource pool, if a preset condition is satisfied, determining an initial virtual machine deployment strategy according to the resource request information, the resource use information and a preset reinforcement learning network, the initial virtual machine deployment strategy relates to at least one initial virtual machine, acquiring a network address of each initial virtual machine, and if the network address of the at least one initial virtual machine in the initial virtual machine deployment cannot be acquired, updating the initial virtual machine deployment strategy according to the resource request information, the updated resource use information and a preset reinforcement learning network, and under the condition that the network addresses of all the initial virtual machines can be obtained, determining the initial virtual machine deployment strategy as a target virtual machine deployment strategy, and performing deployment according to the target virtual machine deployment strategy. According to the virtual machine deployment strategy, the virtual machine can be reasonably deployed.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a virtual machine deployment method, apparatus, and computer equipment. Background Art

[0002] With the development of economy, service-oriented has become an inevitable trend of industrial development. The results of various production activities are gradually being provided to users in the form of services. Cloud computing is an important manifestation of the service-oriented trend and has achieved rapid development around the world with its convenience, economy and easy monitoring. Currently, more and more application service providers and small and medium-sized enterprises choose to deploy applications on cloud platforms.

[0003] However, how to reasonably deploy virtual machines on a cloud platform is one of the main challenges currently facing the cloud computing field. Summary of the invention

[0004] Based on this, it is necessary to provide a virtual machine deployment method, apparatus and computer equipment that can reasonably deploy virtual machines in response to the above technical problems.

[0005] In a first aspect, the present application provides a virtual machine deployment method, which is applied to a resource pool, wherein the resource pool includes a plurality of virtual machines; the method includes:

[0006] Receive resource request information sent by the client and obtain resource usage information of the resource pool;

[0007] When a preset condition is met, determining an initial virtual machine deployment strategy according to the resource request information, the resource usage information and a preset reinforcement learning network, wherein the initial virtual machine deployment strategy involves at least one initial virtual machine;

[0008] Obtaining a network address of each of the initial virtual machines, and if the network address of at least one initial virtual machine cannot be obtained, updating resource usage information of the resource pool, and updating the initial virtual machine deployment strategy according to the resource request information, the updated resource usage information and the preset reinforcement learning network;

[0009] In the case that the network addresses of all the initial virtual machines can be obtained, the initial virtual machine deployment policy is determined as the target virtual machine deployment policy, and deployment is performed according to the target virtual machine deployment policy.

[0010] In one of the embodiments, the resource request information includes a resource request amount, and the resource usage information includes a remaining amount of resources in a resource pool;

[0011] After receiving the resource request information sent by the client and obtaining the resource usage information of the resource pool, the method further includes:

[0012] In the case where the resource request amount is less than or equal to the resource remaining amount, it is determined that the resource pool has sufficient resources and the resource request information meets a preset condition;

[0013] In the case that the resource request amount is greater than the resource remaining amount, it is determined whether the resource request information meets a preset condition according to the resource request information and the resource usage information.

[0014] In one embodiment, the resource request information further includes a request priority, and the resource usage information includes a service priority being processed by the resource pool; and judging whether the resource request information meets a preset condition according to the resource request information and the resource usage information includes:

[0015] In a case where the request priority of the client is higher than the priority of at least one service being processed in the resource pool, determining that the resource request information meets a preset condition;

[0016] When the request priority of the client is lower than the priority of all services being processed in the resource pool, it is determined that the resource request information does not meet the preset condition.

[0017] In one embodiment, the reinforcement learning network includes a prediction network, the resource usage information includes a resource usage status of a resource pool, and the resource request information includes a resource request amount; determining an initial virtual machine deployment strategy according to the virtual machine request information, the resource usage information, and a preset reinforcement learning network includes:

[0018] The resource usage status of the resource pool, the resource request amount, multiple measurement targets and preset measurement target weights are input into the prediction network to obtain the action expectation flow and action advantage flow of multiple candidate virtual machine deployment strategies; wherein the action expectation flow is used to indicate the mean value of the measurement target results of the candidate virtual machine deployment strategies, and the action advantage flow is used to indicate the difference of the measurement target results of each candidate virtual machine deployment strategy;

[0019] An initial virtual machine deployment strategy is determined from a plurality of candidate virtual machine deployment strategies according to each of the action expectation flows and the action advantage flows.

[0020] In one embodiment, determining the initial virtual machine deployment strategy from a plurality of candidate virtual machine deployment strategies according to each of the action expectation flows and the action advantage flows includes:

[0021] Determining a target optimization value of each candidate virtual machine deployment strategy according to each action expectation flow and each action advantage flow;

[0022] An initial virtual machine deployment strategy is determined according to the target optimization value of each candidate virtual machine deployment strategy and a preset measurement target weight.

[0023] In one embodiment, the target virtual machine deployment strategy involves at least one target virtual machine, and the method further includes:

[0024] When there are multiple target virtual machines, tasks are allocated to each of the target virtual machines according to the resource request information and a preset ant colony optimization algorithm.

[0025] In a second aspect, the present application provides a virtual machine deployment device, the device comprising:

[0026] The first acquisition module is used to receive resource request information sent by the client and obtain resource usage information of the resource pool;

[0027] A first determination module is used to determine an initial virtual machine deployment strategy according to the resource request information, the resource usage information and a preset reinforcement learning network when a preset condition is met, wherein the initial virtual machine deployment strategy involves at least one initial virtual machine;

[0028] A second acquisition module is used to obtain the network address of each of the initial virtual machines, and when the network address of at least one initial virtual machine cannot be obtained, update the resource usage information of the resource pool, and update the initial virtual machine deployment strategy according to the resource request information, the updated resource usage information and the preset reinforcement learning network;

[0029] The second determination module is used to determine the initial virtual machine deployment policy as the target virtual machine deployment policy and perform deployment according to the target virtual machine deployment policy when the network addresses of all initial virtual machines can be obtained.

[0030] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the above embodiments when executing the computer program.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of the above embodiments are implemented.

[0032] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of any one of the methods in the above embodiments when executed by a processor.

[0033] The above-mentioned virtual machine deployment method, device and computer equipment, the method includes receiving resource request information sent by the client, and obtaining resource usage information of the resource pool. When it is determined that the preset conditions are met, the initial virtual machine deployment strategy is determined according to the resource request information, the resource usage information and the preset reinforcement learning network. The reinforcement learning network used in this application can determine a suitable initial virtual machine deployment strategy according to the resources required by the client and the current state of the resource pool. The initial virtual machine deployment strategy can maximize the expected satisfaction of the virtual machine or maximize the utilization rate of the resource pool. The initial virtual machine deployment strategy involves at least one initial virtual machine, and the network address of each initial virtual machine is obtained. If the network address of at least one initial virtual machine in the initial virtual machine deployment cannot be obtained, it indicates that the initial virtual machine whose network address cannot be obtained is currently unavailable, so the resource usage information of the resource pool is updated, and then the initial virtual machine deployment strategy is updated according to the resource request information, the updated resource usage information and the preset reinforcement learning network. When the network addresses of all initial virtual machines can be obtained, the initial virtual machine deployment strategy is determined as the target virtual machine deployment strategy, and deployed according to the target virtual machine deployment strategy. Using the virtual machine deployment strategy of this application to respond to the resource request of the client can realize the reasonable deployment of virtual machines and the reasonable scheduling of resources in the resource pool. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] FIG1 is an application environment diagram of a virtual machine deployment method in one embodiment;

[0036] FIG2 is a schematic diagram of a flow chart of a virtual machine deployment method in one embodiment;

[0037] FIG3 is a schematic diagram of a process of virtual machine deployment in another embodiment;

[0038] FIG4 is a schematic diagram of a flow chart of step S302 in one embodiment;

[0039] FIG5 is a schematic diagram of a flow chart of step S202 in one embodiment;

[0040] FIG6 is a schematic diagram of a flow chart of step S502 in one embodiment;

[0041] FIG7 is a structural block diagram of a virtual machine deployment device in one embodiment;

[0042] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] The virtual machine deployment method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, it can be applied to the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0045] In an exemplary embodiment, the present application provides a virtual machine deployment method, which can be executed by a device for executing the method, and the execution device can be implemented by software and / or hardware. The method is applied to a resource pool, which can be a cloud computing platform, and the cloud computing platform can include multiple servers, each of which can have multiple virtual machines. Please refer to Figure 2, the method includes:

[0046] S201: receiving resource request information sent by a client, and obtaining resource usage information of a resource pool.

[0047] Resource request information refers to the amount of resources required by the client, such as computing resources, memory resources, and bandwidth resources requested by the client, and resource usage information of the resource pool refers to the resource status of the resource pool at the current moment when the resource request information sent by the client is received. After the execution device receives the resource request information sent by the client, it can determine whether the current resource pool meets the preset conditions based on the amount of resources required by the client and the resource status of the resource pool at the current moment, that is, whether the current resource pool can respond to the client's request.

[0048] S202: When preset conditions are met, an initial virtual machine deployment strategy is determined according to resource request information, resource usage information, and a preset reinforcement learning network, where the initial virtual machine deployment strategy involves at least one initial virtual machine.

[0049] If it is determined that the current resource pool can respond to the client's request, the execution device can input the client's resource request information and the current resource usage information of the resource pool into a preset reinforcement learning network. The reinforcement learning network is a neural network that uses a value-based Q learning algorithm in the reinforcement learning algorithm and has multi-objective optimization characteristics. The reinforcement learning network may include a current predictor and a target predictor. First, the parameters of the current predictor are updated according to the loss function. After each N rounds of iteration, the parameters of the current predictor are copied to the target predictor. In addition, the samples in the experience pool are also updated, and the network is trained by random processing to reduce the correlation of the data. In this application, by using a reinforcement learning network to determine the virtual machine deployment strategy, the determined initial virtual machine deployment strategy can maximize the expected satisfaction of the virtual machine or maximize the utilization of the resource pool.

[0050] The virtual machine deployment strategy is used to indicate that a virtual machine on a server is allocated to a client to respond to the client's resource request. For example, the cloud computing platform includes server 1, server 2, and server 3. Server 1 has virtual machines 1 to 10, server 2 has virtual machines 11 to 20, and server 3 has virtual machines 21 to 30. The virtual machine deployment strategy can be used to determine that virtual machine 3 on server 1, virtual machines 11 and 13 on server 2, and virtual machine 26 on server 3 respond to client requirements.

[0051] S203: Obtain the network address of each initial virtual machine. If the network address of at least one initial virtual machine cannot be obtained, update the resource usage information of the resource pool, and update the initial virtual machine deployment strategy according to the resource request information, the updated resource usage information and the preset reinforcement learning network.

[0052] After the initial virtual machine deployment strategy is determined, the network address of each server can be collected, and the network address of the client can be marked on each initial virtual machine. If the location information of a server and the virtual machine running on it is not visible, the resource usage information of the resource pool needs to be updated and the initial virtual machine deployment strategy needs to be re-determined to ensure the performance of the resource pool and the reliable execution of resource requests.

[0053] S204: When the network addresses of all initial virtual machines can be obtained, the initial virtual machine deployment policy is determined as the target virtual machine deployment policy, and deployment is performed according to the target virtual machine deployment policy.

[0054] If the network addresses of all initial virtual machines involved in the initial virtual machine deployment strategy are visible, that is, the network addresses of the servers where all initial virtual machines involved in the initial virtual machine deployment strategy are located are visible, then the network address of the client can be re-marked on each initial virtual machine, and the initial virtual machine deployment strategy can be determined as the target virtual machine deployment strategy, and each initial virtual machine can be determined as the target virtual machine and deployed according to the target virtual machine deployment strategy.

[0055] The above-mentioned virtual machine deployment method, device and computer equipment, the method includes receiving resource request information sent by the client, and obtaining resource usage information of the resource pool. When it is determined that the preset conditions are met, the initial virtual machine deployment strategy is determined according to the resource request information, the resource usage information and the preset reinforcement learning network. The reinforcement learning network used in this application can determine a suitable initial virtual machine deployment strategy according to the resources required by the client and the current state of the resource pool. The initial virtual machine deployment strategy can maximize the expected satisfaction of the virtual machine or maximize the utilization rate of the resource pool. The initial virtual machine deployment strategy involves at least one initial virtual machine, and the network address of each initial virtual machine is obtained. If the network address of at least one initial virtual machine in the initial virtual machine deployment cannot be obtained, it indicates that the initial virtual machine whose network address cannot be obtained is currently unavailable, so the resource usage information of the resource pool is updated, and then the initial virtual machine deployment strategy is updated according to the resource request information, the updated resource usage information and the preset reinforcement learning network. When the network addresses of all initial virtual machines can be obtained, the initial virtual machine deployment strategy is determined as the target virtual machine deployment strategy, and deployed according to the target virtual machine deployment strategy. Using the virtual machine deployment strategy of this application to respond to the resource request of the client can achieve reasonable scheduling of resources in the resource pool.

[0056] In an exemplary embodiment, please refer to FIG. 3 , in step S201 , after receiving resource request information sent by the client and acquiring resource usage information of the resource pool, the method further includes steps S301 and S302 .

[0057] S301: When the resource request amount is less than or equal to the resource remaining amount, it is determined that the resource pool resources are sufficient and the resource request information meets the preset conditions.

[0058] In this embodiment, the resource request information includes the resource request amount, and the resource usage information includes the resource remaining amount of the resource pool. After receiving the resource request information of the client and obtaining the resource usage information of the resource pool, it is necessary to parse the resource request information of the client to analyze the resource request amount required by the client, and parse the resource usage information of the resource pool to analyze the resource remaining amount of the resource pool, and then compare the resource request amount with the resource remaining amount. If the resource request amount is less than or equal to the resource remaining amount, it indicates that the resource pool currently has sufficient resources and can immediately respond to the client's request.

[0059] S302: When the resource request amount is greater than the resource remaining amount, determine whether the resource request information meets a preset condition based on the resource request information and the resource usage information.

[0060] If the resource request amount exceeds the current response capacity of the resource pool, it is necessary to further determine whether to respond to the client's request immediately based on the resource request information and resource usage information.

[0061] In an exemplary embodiment, please refer to FIG. 4 , step S302 , judging whether the resource request information meets a preset condition according to the resource request information and the resource usage information, includes step S401 and step S402 .

[0062] S401: When the request priority of the client is higher than the priority of at least one service being processed in the resource pool, it is determined that the resource request information meets a preset condition.

[0063] Resource request information also includes request priority. When the amount of resource requests exceeds the current response capacity of the resource pool, it needs to be processed according to the priority of the service. High-priority requests are urgent. The execution device needs to further parse the resource usage information to analyze the service priority being processed by the resource pool, that is, the priority of the service being processed on each virtual machine in the resource pool at the current moment. If the client's request priority is higher than the priority of some services being processed in the resource pool, all services with a lower request priority will be made transparent, and the virtual machines with services with a lower request priority will be preempted.

[0064] S402: When the request priority of the client is lower than the priority of all services being processed in the resource pool, it is determined that the resource request information does not meet a preset condition.

[0065] If the client's request priority is lower than the priority of all services being processed in the resource pool, due to the excessive network load, in order to ensure the stability of the cloud computing platform, the execution device can temporarily reject the client's request and make it wait in the queue until the network is idle, that is, when the resource pool has sufficient resources, it will be deployed and executed.

[0066] In an exemplary embodiment, please refer to FIG. 5 , step S202 , determining an initial virtual machine deployment strategy according to virtual machine request information, resource usage information and a preset reinforcement learning network, includes steps S501 and S502 .

[0067] S501: Inputting the resource usage status, resource request amount, multiple measurement targets and preset measurement target weights of the resource pool into a prediction network to obtain action expectation flows and action advantage flows of multiple candidate virtual machine deployment strategies.

[0068] In this embodiment, the reinforcement learning network may include a prediction network, the resource usage information includes the resource usage status of the resource pool, and the resource request information includes the resource request amount. The resource usage status of the current resource pool, the measurement target to be measured, and the target proportion vector may be input to the prediction network. Among them, the resource usage status may be the resource usage status of each server in the resource pool, and the multiple measurement targets may include virtual machine reliability, resource pool utilization, and load balancing value, and the target proportion vector is the preset weight of each measurement target. After inputting the resource usage status of the current resource pool, the measurement target to be measured, and the target proportion vector to the prediction network, the prediction network can analyze all possible virtual machine deployment strategies under the current usage status of the resource pool, that is, the candidate virtual machine deployment strategy, and evaluate each candidate virtual machine deployment strategy, and output the action expectation flow and action advantage flow of each candidate virtual machine deployment strategy. Among them, the action expectation flow represents the mean value of the measurement target result of the candidate virtual machine deployment strategy, and the action advantage flow represents the difference in the measurement target result of each candidate virtual machine deployment strategy. In the application, the output of the action advantage flow can be normalized to ensure that the predicted average value of the action advantage flow is zero, so that the initial virtual machine deployment strategy finally output can be better represented by the action expectation flow.

[0069] S502: Determine an initial virtual machine deployment strategy from a plurality of candidate virtual machine deployment strategies according to each action expectation flow and action advantage flow.

[0070] After obtaining the action expectation flow and the action advantage flow, each candidate virtual machine deployment strategy can be analyzed to determine the initial virtual machine deployment strategy.

[0071] In an exemplary embodiment, please refer to FIG. 6 , step S502 , determining an initial virtual machine deployment strategy from a plurality of candidate virtual machine deployment strategies according to each action expectation flow and action advantage flow, includes step S601 and step S602 .

[0072] S601: Determine the target optimization value of each candidate virtual machine deployment strategy according to each action expectation flow and each action advantage flow.

[0073] In the application, the action expectation flow and the action advantage flow of each candidate virtual machine deployment strategy can be added together, and the sum of the action expectation flow and the action advantage flow can be determined as the target optimization value of the candidate virtual machine deployment strategy.

[0074] S602: Determine an initial virtual machine deployment strategy according to the target optimization value of each candidate virtual machine deployment strategy and a preset measurement target weight.

[0075] After obtaining the target optimization value of each candidate virtual machine deployment strategy, the reinforcement learning network can multiply the target optimization value with the input preset measurement target weight as the final result of each candidate virtual machine deployment strategy, and the action with the largest candidate virtual machine deployment strategy is the optimal action for this virtual machine deployment, that is, the initial virtual machine deployment strategy.

[0076] In an exemplary embodiment, the target virtual machine deployment strategy involves at least one target virtual machine, and the method further includes the step of allocating tasks to each target virtual machine according to resource request information and a preset ant colony optimization algorithm when there are multiple target virtual machines.

[0077] In this embodiment, the preset ant colony optimization algorithm is an improved ant colony optimization algorithm combined with a simulated annealing algorithm. The ant colony optimization algorithm (Ant Colony Optimization, ACO) has high flexibility and robustness, but there are defects such as slow convergence execution rate and easy to fall into local optimum within the algorithm, so the algorithm needs to be gradually improved and continuously improved. Therefore, this application proposes an improved ant colony simulated annealing algorithm in combination with the characteristics of the information network, so that it can better serve the space-based information network and achieve the expected goal of resource scheduling.

[0078] The specific implementation steps are as follows:

[0079] First, initialize the parameters. Analyze the task requirements and attributes provided by the user, and compare the characteristic attribute capabilities of the resources to pheromones. The initial value of the pheromone for path search is determined by the attribute capabilities of the resources and the location information of the resources. The specific calculation method is as follows: , Indicates the attribute capabilities of resources. Indicates the location relationship of resources.

[0080] The second is the way ants choose resources. They consider the location and the credibility of resources comprehensively, and then calculate the transfer probability through pheromone and heuristic information to select the next resource. At time t, the migration probability of task Ti, that is, ant k, between resources Rj is expressed as To represent it, as follows:

[0081]

[0082] In the formula, is the pheromone concentration of task Ti scheduled to resource Rj for execution, and All are inspirational factors. Indicates the location distance. Indicates the degree of credibility. These two coefficients represent the weight index of the control pheromone and the state transition probability respectively. The smaller it is, the closer the task requirement area is to the resource service range. The larger the value, the higher the trustworthiness of the resource. When the ant finishes searching for a resource, it removes it from the Remove from the.

[0083] Finally, the pheromone is updated. When the ant colony selects a path, it will be affected by the concentration of pheromones to select the best path. The concentration of pheromones will change all the time during the continuous iteration process, and tasks need to be deployed on the path with a higher pheromone concentration. The time constraint function is improved into the pheromone update rule. The following formula evaluates the time cost of the selected resource scheduling scheme "K".

[0084]

[0085] The time constraint function is proportional to the time required to complete the task. The smaller the value, the shorter the execution time of the task group. The design of needs to consider the task completion execution time in the space-based information network resource scheduling process, so the pheromone increment The calculation method is as follows:

[0086]

[0087] in, is a constant, the pheromone increment refers to the increment used when the local pheromone is updated, and in the complete algorithm iteration process, it is the optimal path that can be found in all search paths after all ants have completed their own path searches.

[0088] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0089] The virtual machine deployment method of the present application is mainly designed to solve the problem of dynamic network changes. In the information network environment, service requests always change with the time and spatial location of access. Most requests can be deployed on a server within a single visible time window, but as the distance changes, the location of the server and the location visibility of the virtual machine deployment requester change. When the visible time window changes, migrating the virtual machine to a server visible to the user is an effective measure to ensure the efficient and reliable operation of the virtual machine.

[0090] Based on the same inventive concept, the embodiment of the present application also provides a virtual machine deployment device for implementing the virtual machine deployment method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more virtual machine deployment device embodiments provided below can refer to the limitations of the virtual machine deployment method above, and will not be repeated here.

[0091] In an exemplary embodiment, referring to FIG. 7 , the present application provides a virtual machine deployment device, the device comprising:

[0092] The first acquisition module 701 is used to receive resource request information sent by the client and obtain resource usage information of the resource pool;

[0093] A first determination module 702 is used to determine an initial virtual machine deployment strategy according to resource request information, resource usage information and a preset reinforcement learning network when a preset condition is met, where the initial virtual machine deployment strategy involves at least one initial virtual machine;

[0094] The second acquisition module 703 is used to obtain the network address of each initial virtual machine, and when the network address of at least one initial virtual machine cannot be obtained, update the resource usage information of the resource pool, and update the initial virtual machine deployment strategy according to the resource request information, the updated resource usage information and the preset reinforcement learning network;

[0095] The second determination module 704 is used to determine the initial virtual machine deployment policy as the target virtual machine deployment policy and perform deployment according to the target virtual machine deployment policy when the network addresses of all initial virtual machines can be obtained.

[0096] In one embodiment, the resource request information includes a resource request amount, and the resource usage information includes a resource remaining amount in the resource pool; the virtual machine deployment device further includes:

[0097] The first judgment module is used to judge that the resource pool resources are sufficient and the resource request information meets the preset conditions when the resource request amount is less than or equal to the resource remaining amount;

[0098] The second judgment module is used to judge whether the resource request information meets a preset condition according to the resource request information and the resource usage information when the resource request amount is greater than the resource remaining amount.

[0099] In one embodiment, the resource request information further includes a request priority, and the resource usage information includes a service priority being processed by the resource pool; the second judgment module includes:

[0100] A first judgment submodule, configured to judge that the resource request information meets a preset condition when the request priority of the client is higher than the priority of at least one service being processed in the resource pool;

[0101] The second judgment submodule is used to judge that the resource request information does not meet the preset conditions when the request priority of the client is lower than the priority of all services being processed in the resource pool.

[0102] In one embodiment, the reinforcement learning network includes a prediction network, the resource usage information includes a resource usage status of a resource pool, and the resource request information includes a resource request amount; the first determination module includes:

[0103] The first determination submodule is used to input the resource usage status, resource request amount, multiple measurement targets and preset measurement target weights of the resource pool into the prediction network to obtain the action expectation flow and action advantage flow of multiple candidate virtual machine deployment strategies; wherein the action expectation flow is used to indicate the mean value of the measurement target results of the candidate virtual machine deployment strategies, and the action advantage flow is used to indicate the difference of the measurement target results of each candidate virtual machine deployment strategy;

[0104] The second determination submodule is used to determine an initial virtual machine deployment strategy from a plurality of candidate virtual machine deployment strategies according to each action expectation flow and action advantage flow.

[0105] In one embodiment, the second determining submodule includes:

[0106] A first determination unit is used to determine a target optimization value of each candidate virtual machine deployment strategy according to each action expectation flow and each action advantage flow;

[0107] The second determination unit is used to determine the initial virtual machine deployment strategy according to the target optimization value of each candidate virtual machine deployment strategy and the preset measurement target weight.

[0108] In one embodiment, the target virtual machine deployment strategy involves at least one target virtual machine, and the virtual machine deployment device further includes:

[0109] The task allocation module is used to allocate tasks to each target virtual machine according to resource request information and a preset ant colony optimization algorithm when there are multiple target virtual machines.

[0110] Each module in the above-mentioned virtual machine deployment device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0111] In an exemplary embodiment, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods in the above embodiments when executing the computer program.

[0112] The computer device may be a terminal, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a virtual machine deployment method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0113] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0114] In an exemplary embodiment, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0115] In an exemplary embodiment, the present application provides a computer program product, including a computer program, which implements the steps of any of the methods in the above embodiments when the computer program is executed by a processor. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant regulations.

[0116] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0118] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A virtual machine deployment method, characterized in that: Applied to a resource pool, the resource pool includes a plurality of virtual machines; the method includes: Receive resource request information sent by the client and obtain resource usage information of the resource pool; Under the condition that a preset condition is met, determining an initial virtual machine deployment strategy according to the resource request information, the resource usage information and a preset reinforcement learning network, wherein the initial virtual machine deployment strategy involves at least one initial virtual machine; Obtaining a network address of each of the initial virtual machines, and updating resource usage information of the resource pool when the network address of at least one initial virtual machine cannot be obtained, and updating the initial virtual machine deployment strategy according to the resource request information, the updated resource usage information and the preset reinforcement learning network; In the case that the network addresses of all the initial virtual machines can be obtained, the initial virtual machine deployment policy is determined as the target virtual machine deployment policy, and deployment is performed according to the target virtual machine deployment policy.

2. The virtual machine deployment method according to claim 1, characterized in that: The resource request information includes a resource request amount, and the resource usage information includes a resource remaining amount in a resource pool; After receiving the resource request information sent by the client and obtaining the resource usage information of the resource pool, the method further includes: In the case where the resource request amount is less than or equal to the resource remaining amount, it is determined that the resource pool has sufficient resources and the resource request information meets a preset condition; In the case that the resource request amount is greater than the resource remaining amount, it is determined whether the resource request information meets a preset condition according to the resource request information and the resource usage information.

3. The virtual machine deployment method according to claim 2, characterized in that: The resource request information further includes a request priority, and the resource usage information includes a service priority being processed by the resource pool; and judging whether the resource request information meets a preset condition according to the resource request information and the resource usage information includes: In a case where the request priority of the client is higher than the priority of at least one service being processed in the resource pool, determining that the resource request information meets a preset condition; When the request priority of the client is lower than the priority of all services being processed in the resource pool, it is determined that the resource request information does not meet the preset condition.

4. The virtual machine deployment method according to claim 1, characterized in that: The reinforcement learning network includes a prediction network, the resource usage information includes a resource usage status of a resource pool, and the resource request information includes a resource request amount; determining an initial virtual machine deployment strategy according to the virtual machine request information, the resource usage information and a preset reinforcement learning network includes: The resource usage status of the resource pool, the resource request amount, multiple measurement targets and preset measurement target weights are input into the prediction network to obtain the action expectation flow and action advantage flow of multiple candidate virtual machine deployment strategies; wherein the action expectation flow is used to indicate the mean value of the measurement target results of the candidate virtual machine deployment strategies, and the action advantage flow is used to indicate the difference of the measurement target results of each candidate virtual machine deployment strategy; An initial virtual machine deployment strategy is determined from a plurality of candidate virtual machine deployment strategies according to each of the action expectation flows and the action advantage flows.

5. The virtual machine deployment method according to claim 4, characterized in that: The step of determining an initial virtual machine deployment strategy from a plurality of candidate virtual machine deployment strategies according to each of the action expectation flows and the action advantage flows includes: Determining a target optimization value of each candidate virtual machine deployment strategy according to each action expectation flow and each action advantage flow; An initial virtual machine deployment strategy is determined according to the target optimization value of each candidate virtual machine deployment strategy and a preset measurement target weight.

6. The virtual machine deployment method according to claim 1, characterized in that: The target virtual machine deployment strategy involves at least one target virtual machine, and the method further includes: When there are multiple target virtual machines, tasks are allocated to each of the target virtual machines according to the resource request information and a preset ant colony optimization algorithm.

7. A virtual machine deployment device, characterized in that: The device comprises: The first acquisition module is used to receive resource request information sent by the client and obtain resource usage information of the resource pool; A first determination module is used to determine an initial virtual machine deployment strategy according to the resource request information, the resource usage information and a preset reinforcement learning network when a preset condition is met, wherein the initial virtual machine deployment strategy involves at least one initial virtual machine; A second acquisition module is used to obtain the network address of each of the initial virtual machines, and when the network address of at least one initial virtual machine cannot be obtained, update the resource usage information of the resource pool, and update the initial virtual machine deployment strategy according to the resource request information, the updated resource usage information and the preset reinforcement learning network; The second determination module is used to determine the initial virtual machine deployment policy as the target virtual machine deployment policy and perform deployment according to the target virtual machine deployment policy when the network addresses of all initial virtual machines can be obtained.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images