Virtual machine scheduling method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of resource scheduling, and discloses a virtual machine scheduling method and device, computer equipment and a storage medium, and the method comprises the steps: determining an energy consumption coefficient and a service quality coefficient during the operation of a virtual machine based on the state information of a virtual machine resource, and constructing a virtual machine scheduling task target; therefore, the influence of energy consumption and service quality is considered in the scheduling process, and the resource utilization rate and the system performance are improved. And then, according to the state information and a virtual machine scheduling task target, constructing a decision space, and pre-sequencing a task request queue by utilizing a trained task sequencing model, so that the scheduling efficiency of a subsequent scheduling process is improved. And finally, determining a virtual machine scheduling strategy by using the trained virtual machine resource scheduling model and the decision space, and allocating virtual machine resources to the sorted task request sorting queue, thereby realizing reasonable virtual machine scheduling.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and in particular to a virtual machine scheduling method, device, computer equipment and storage medium. Background Art

[0002] In the field of cloud computing, virtual machine scheduling is a critical task that determines how virtual machines are deployed on physical servers to maximize resource utilization and performance.

[0003] Traditional static rule-based virtual machine scheduling algorithms usually only consider one or a few factors, such as task duration, system load, etc., and consider a single problem, ignoring the impact of other factors such as task execution time. With the continuous development of cloud computing and big data technology, the amount of data and computing is increasing, and the system load is also increasing. The use of traditional static rule-based algorithms may cause some important and special tasks to be delayed and unable to be responded to in time. Although the virtual machine scheduling algorithm based on deep learning can solve the problem of single consideration of static rule algorithms, it does not consider the resource usage in the virtual machine scheduling process and the relationship between dynamic changes and static factors. There are still certain limitations, the scheduling efficiency is not high and often does not match the actual resource usage. Summary of the invention

[0004] In view of this, the present invention provides a virtual machine scheduling method, apparatus, computer equipment and storage medium to solve the problems that the existing virtual machine scheduling algorithm based on deep learning does not consider the usage of resources in the virtual machine scheduling process and the relationship between dynamic changes and static factors, and the scheduling strategy is unreasonable and the scheduling efficiency is low.

[0005] In a first aspect, the present invention provides a virtual machine scheduling method, the method comprising:

[0006] Obtain user task requests and construct a task request queue, as well as obtain status information of virtual machine resources;

[0007] Determine the energy consumption coefficient and service quality coefficient of the virtual machine when it is running based on the state information, and obtain the virtual machine scheduling task target according to the energy consumption coefficient and service quality coefficient;

[0008] Construct a decision space based on the state information and the virtual machine scheduling task objectives;

[0009] According to the task request queue and the trained task sorting model, a task request sorting queue is obtained;

[0010] The virtual machine scheduling strategy is obtained by using the task request sorting queue, decision space and the trained virtual machine resource scheduling model.

[0011] Beneficial effects: The present invention determines the energy consumption coefficient and service quality coefficient of the virtual machine during operation based on the state information of the virtual machine resources, and then constructs the virtual machine scheduling task target, so as to consider the impact of energy consumption and service quality in the scheduling process, and facilitates improving resource utilization and system performance. Then, according to the state information and the virtual machine scheduling task target, a decision space is constructed, and the task request queue is pre-sorted using the trained task sorting model to improve the scheduling efficiency of the subsequent scheduling process. Finally, the virtual machine scheduling strategy is determined using the trained virtual machine resource scheduling model and decision space, and virtual machine resources are allocated to the sorted task request sorting queue to achieve reasonable virtual machine scheduling.

[0012] In some optional implementations, determining the energy consumption coefficient and the service quality coefficient of the virtual machine when it is running based on the state information includes:

[0013] Based on the status information, the network bandwidth, the memory occupied by the virtual machine, the maximum energy consumption, the idle probability, and the CPU utilization and memory utilization of the virtual machine when running are obtained;

[0014] The physical machine power is calculated based on the maximum energy consumption, idle probability, CPU utilization, and memory utilization, and the energy consumption coefficient of the virtual machine when running is determined based on the physical machine power;

[0015] The virtual machine jam coefficient is calculated based on the CPU utilization and memory utilization, and the virtual machine migration time is calculated based on the network bandwidth and occupied memory. The service quality coefficient of the virtual machine during operation is determined based on the virtual machine jam coefficient and the virtual machine migration time.

[0016] Beneficial effect: During the virtual machine scheduling process, the present invention determines the energy consumption of the virtual machine during operation through the utilization of resources such as the central processing unit and memory. On the other hand, it uses the virtual machine jam coefficient to measure the virtual machine jam phenomenon caused by resource utilization, and considers the impact of migration on performance when the virtual machine jams and fails. The virtual machine jam coefficient and virtual machine migration time are used to measure the service quality of the virtual machine, reasonably use dynamically changing data, consider energy consumption and service coefficient issues, and optimize the virtual machine scheduling process.

[0017] In some optional implementations, a decision space is constructed according to the state information and the virtual machine scheduling task target, including:

[0018] Constructing a state set according to the remaining resource quantity, current task demand quantity and current task total quantity corresponding to various virtual machine resources in the state information; wherein the virtual machine resources include at least some of the following items: central processing unit, memory and bandwidth;

[0019] Constructing an action set according to a preset execution instruction; wherein the preset execution instruction includes at least some of the following items: starting a virtual machine, shutting down a virtual machine, migrating a virtual machine, adding a virtual machine, reducing a virtual machine, and not executing an action;

[0020] Combine the state set and the action set to obtain multiple state transition combinations, and calculate the reward coefficient corresponding to each state transition combination according to the virtual machine scheduling task target; wherein the state transition combination includes the current state of the virtual machine, the current action, and the updated state after executing the current action;

[0021] According to the state set, action set, state transition combination and corresponding reward coefficient, the decision space for virtual machine scheduling is constructed.

[0022] Beneficial effects: The present invention uses the remaining resource quantity, current task demand quantity and current task total quantity corresponding to the virtual machine resources to determine the environmental state of the virtual machine to construct a state set, and constructs an action set according to the instructions that the virtual machine may execute. Then, the state and action are combined, and the reward feedback after the virtual machine executes the action is calculated to determine the state transition strategy, reward set and feedback. Finally, according to the five-tuple consisting of the state set, action set, state transition strategy, reward set and feedback, the decision space for virtual machine scheduling is constructed to facilitate the determination of the virtual machine scheduling strategy.

[0023] In some optional implementations, obtaining a task request sorting queue according to the task request queue and the trained task sorting model includes:

[0024] Determine a first attribute parameter of each user task request in the task request queue; wherein the first attribute parameter includes at least part of the following items: task priority, arrival time, processing time and task type;

[0025] The first attribute parameter and the trained task sorting model are used to sort the execution order of each user task request in the task request queue to obtain a task request sorting queue.

[0026] Beneficial effects: The present invention utilizes a trained task sorting model and combines multiple global data such as task priority, arrival time, processing time, task type, etc. of user task requests to sort the task request queue, solving the problem of single consideration when sorting request tasks in traditional methods. By more reasonably sorting tasks before virtual machine scheduling, it is beneficial to improve virtual machine scheduling efficiency.

[0027] In some optional implementations, the training process of the task sequencing model includes:

[0028] Construct a task data set and add a label to each scheduled task in the task data set; the second attribute parameter of the scheduled task includes task priority, arrival time, processing time, waiting time from queuing to completion, and task type. The label is obtained by weighted calculation based on the waiting time and task priority.

[0029] A first network model is constructed based on the first preset model structure, and the first network model is trained using the task data set to obtain a second network model; wherein the output result of the second network model is the execution order and task completion time of the scheduled tasks;

[0030] The predicted label of the scheduled task is calculated according to the execution order, task completion time and task priority, and the first loss value of the second network model is calculated according to the label and the predicted label, and the model parameters of the second network model are adjusted based on the first loss value until the model converges to obtain a trained task sorting model.

[0031] Beneficial effects: The present invention constructs a data set using global data such as task priority, arrival time, processing time, waiting time from queuing to completion, and task type of the scheduled tasks, and trains the task sorting model, so that the trained task sorting model can consider multiple factors when sorting user task requests, thereby improving the rationality of task sorting, reducing resource waste caused by unreasonable sorting, facilitating virtual machine scheduling to process tasks in a reasonable order, and improving user experience.

[0032] In some optional implementations, the task request sorting queue, the decision space, and the trained virtual machine resource scheduling model are used to obtain a virtual machine scheduling strategy, including:

[0033] Input the task request sorting queue and decision space into the trained virtual machine resource scheduling model, predict the reward value of the virtual machine executing the corresponding action in different states, and obtain the reward prediction result; wherein the decision space includes the state set, action set, state transition, reward set and return set;

[0034] According to the reward prediction results, the virtual machine scheduling strategy corresponding to the task request sorting queue is obtained.

[0035] Beneficial effect: The present invention uses the trained virtual machine resource scheduling model and decision space to determine the virtual machine scheduling strategy, and allocates virtual machine resources to user task requests in the task request sorting queue in order, thereby improving scheduling efficiency and improving user experience.

[0036] In some optional implementations, the training process of the virtual machine resource scheduling model includes:

[0037] The user request process and virtual machine resources are simulated using the simulation environment to obtain simulation status information of the simulation task request queue and virtual machine resources;

[0038] The simulated task request queue is sorted based on the trained task sorting model to obtain the simulated task request sorting queue, and an experience pool is constructed according to the simulation state information; wherein the experience pool includes the reward value when the virtual machine performs the corresponding action in different states;

[0039] A first scheduling model is constructed based on a second preset model structure, and the first scheduling model is trained using a simulated task request sorting queue and an experience pool to obtain a second scheduling model; wherein an output result of the second scheduling model is a maximum reward prediction value when the virtual machine performs corresponding actions in different states;

[0040] A second loss value of the second scheduling model is calculated, and model parameters of the second scheduling model are adjusted based on the second loss value until the model converges, thereby obtaining a trained virtual machine resource scheduling model.

[0041] Beneficial effect: The present invention utilizes a simulation environment to simulate the arrival process of user requests and various virtual machine resources, and then trains the scheduling model according to the simulation data, so that the trained virtual machine resource scheduling model can calculate the reward value after executing the action based on the state set and the action set, thereby determining the scheduling strategy for the virtual machine according to the maximum reward value.

[0042] In a second aspect, the present invention provides a virtual machine scheduling device, the device comprising:

[0043] An acquisition module is used to acquire user task requests and construct a task request queue, as well as to acquire status information of virtual machine resources;

[0044] The first processing module is used to determine the energy consumption coefficient and the service quality coefficient of the virtual machine when it is running based on the state information, and obtain the virtual machine scheduling task target according to the energy consumption coefficient and the service quality coefficient;

[0045] The second processing module is used to construct a decision space according to the state information and the virtual machine scheduling task target;

[0046] The third processing module is used to obtain a task request sorting queue according to the task request queue and the trained task sorting model;

[0047] The fourth processing module is used to obtain a virtual machine scheduling strategy by using the task request sorting queue, the decision space and the trained virtual machine resource scheduling model.

[0048] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the virtual machine scheduling method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the virtual machine scheduling method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 is a flow chart of a virtual machine scheduling method according to an embodiment of the present invention;

[0052] Figure 2 is a flow chart of another virtual machine scheduling method according to an embodiment of the present invention;

[0053] Figure 3 is a training flow chart of a task sequencing model according to an embodiment of the present invention;

[0054] Figure 4 is a training flow chart of a virtual machine resource scheduling model according to an embodiment of the present invention;

[0055] Figure 5 is a flow chart of another virtual machine scheduling method according to an embodiment of the present invention;

[0056] Figure 6 is a structural block diagram of a virtual machine scheduling device according to an embodiment of the present invention;

[0057] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0059] With the rapid development of technologies such as cloud computing, big data, and artificial intelligence, cloud computing has become one of the important research directions in the field of computer technology today. In the field of cloud computing, virtual machine scheduling is a key task. It determines how to deploy virtual machines to physical servers to maximize resource utilization and performance. The current virtual machine scheduling methods have the following main shortcomings:

[0060] 1) Traditional algorithms based on static rules consider only one problem. These algorithms usually only consider one or a few factors, such as task duration, system load, etc., while ignoring the influence of other factors. For example, the First Come First Served (FCFS) algorithm is a virtual machine scheduling algorithm based on static rules. It schedules tasks according to the order of their arrival and only considers the execution time of the tasks, without considering other factors, such as the available resources of the system, the priority of the tasks, etc. This will cause some virtual machines with longer task execution times to wait for a long time, while some virtual machines with shorter execution times will not be scheduled in time, resulting in a waste of resources. With the continuous development of cloud computing and big data technology, the amount of data and computing power is increasing, and the system load is also increasing. The use of traditional algorithms based on static rules will cause some important and special tasks to be delayed and thus fail to be responded to in time.

[0061] 2) Existing deep learning-based algorithms can be divided into: static methods that use request task sorting to improve virtual machine scheduling and dynamic methods that use dynamic data in scheduling to timely solve local optimal solutions. The former simply considers certain factors for permutation and combination, sends the corresponding historical data into the learning model for training to obtain the corresponding model, and lets the model determine the execution order of tasks, without considering the changes in resources during the scheduling process; the latter, for example, uses reinforcement learning in the scheduling process, but only makes dynamic adjustments based on the actual situation of resource usage, and even if task arrival time, priority, etc. are introduced, static data is not effectively used.

[0062] In summary, the existing deep learning-based algorithms have solved the problem of single considerations in static rule-based algorithms, but there are still certain limitations. The static method based on deep learning only considers the reasonable use of data and the dynamic changes in the virtual machine scheduling process through the accumulation of various static factors; the dynamic method based on reinforcement learning only considers the dynamic utilization changes but does not consider the impact of static factors on scheduling; and in the reinforcement learning process, due to the unreasonable specification of the reward function, the resulting virtual machine scheduling algorithm fails to comprehensively consider the energy consumption and service coefficient issues.

[0063] According to an embodiment of the present invention, an embodiment of a virtual machine scheduling method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0064] In this embodiment, a virtual machine scheduling method is provided, which can be used for a device for virtual machine scheduling, such as a computer, etc. Figure 1 is a flowchart of a virtual machine scheduling method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0065] Step S101, obtaining a user task request and constructing a task request queue, and obtaining status information of virtual machine resources.

[0066] Specifically, the Cloud Computing Center uses physical machines and virtual machines to provide cloud computing services to users. Users access the services provided by the Cloud Computing Center through the Internet, such as renting virtual machines (IaaS mode) to run their own applications, or using the development platform provided by the Cloud Computing Center (PaaS mode) for software development, or directly using the software applications provided by the Cloud Computing Center (SaaS mode). Therefore, the Cloud Computing Center can obtain user task requests and build a task request queue based on the arrival time of the user task requests.

[0067] Specifically, the cloud data center can uniformly manage physical machines and virtual machines, and monitor the operating status of physical machines and virtual machines through management software, including hardware health status, resource utilization, etc., so as to obtain status information of various virtual machine resources. It should be noted that the virtual machine resources in the present invention refer to the physical resources that the virtual machine relies on during operation, such as central processing unit (CPU), memory, storage, network, etc., and the present invention is not limited to this.

[0068] Step S102, determining the energy consumption coefficient and service quality coefficient of the virtual machine when it is running based on the state information, and obtaining the virtual machine scheduling task target according to the energy consumption coefficient and service quality coefficient.

[0069] Specifically, virtual machine scheduling refers to allocating virtual machines to different users or tasks based on certain strategies and algorithms to achieve optimal resource utilization and on-time completion of tasks. Various factors are considered when scheduling virtual machines. For example, the cloud computing center monitors in real time that the CPU utilization of a virtual machine on a physical machine is too high. Then, through the dynamic resource allocation mechanism, the virtual machine is migrated to other physical machines with idle resources or its resource allocation is adjusted to ensure service quality.

[0070] Specifically, the present invention measures the energy consumption coefficient and service quality coefficient of the virtual machine during operation according to the state information such as resource utilization, and constructs the virtual machine scheduling task target. The virtual machine scheduling task is a task in which each independent user in a cloud data center (such as an IaaS environment) composed of multiple physical resources submits a request for a number of heterogeneous virtual machines. The virtual machine scheduling task target is constructed to reasonably utilize the dynamically changing data in the scheduling process, to take into account the energy consumption and service coefficient issues, and to maximize the resource utilization and performance.

[0071] Step S103, constructing a decision space according to the state information and the virtual machine scheduling task target.

[0072] Specifically, according to the state information of virtual machine resources, the state set, action set and state transition strategy are determined, and the reward function is better constructed according to the energy consumption coefficient and service quality coefficient in the virtual machine scheduling task objectives, and then the decision space is constructed so that the virtual machine resource scheduling model can reasonably utilize dynamically changing data and consider the energy consumption and service coefficient issues more in the scheduling process.

[0073] Step S104, obtaining a task request sorting queue according to the task request queue and the trained task sorting model.

[0074] Specifically, each user task request in the task request queue has some relevant attribute information, such as task priority, task type, arrival time, processing time, required resources, etc. These attribute information are combined and the trained task sorting model is used to pre-sort the task request queue to obtain the sorted task request sorting queue, which takes into account the impact of various static factors on scheduling, so as to improve scheduling efficiency.

[0075] Step S105, using the task request sorting queue, the decision space and the trained virtual machine resource scheduling model to obtain the virtual machine scheduling strategy.

[0076] Specifically, the task request sorting queue has pre-sorted the user task requests, so that the virtual machine resource scheduling model can schedule the user task requests in a reasonable order, improve the scheduling strategy, and determine the scheduling strategy for each user task request based on the decision space, thereby allocating an adaptive number and type of virtual machines to the task and realizing virtual machine scheduling.

[0077] The virtual machine scheduling method provided in this embodiment determines the energy consumption coefficient and service quality coefficient of the virtual machine when it is running based on the state information of the virtual machine resources, and then constructs the virtual machine scheduling task target, so as to consider the impact of energy consumption and service quality in the scheduling process, so as to improve resource utilization and system performance. Then, according to the state information and the virtual machine scheduling task target, a decision space is constructed, and the task request queue is pre-sorted using the trained task sorting model to improve the scheduling efficiency of the subsequent scheduling process. Finally, the virtual machine scheduling strategy is determined using the trained virtual machine resource scheduling model and decision space, and virtual machine resources are allocated to the sorted task request sorting queue to achieve reasonable virtual machine scheduling.

[0078] In this embodiment, a virtual machine scheduling method is provided, which can be used for a device for virtual machine scheduling, such as a computer, etc. Figure 2 is a flowchart of a virtual machine scheduling method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0079] Step S201, obtain user task request and construct task request queue, and obtain virtual machine resource status information. Figure 1 The description of step S101 of the illustrated embodiment will not be repeated here.

[0080] Step S202, determining the energy consumption coefficient and service quality coefficient of the virtual machine when it is running based on the state information, and obtaining the virtual machine scheduling task target according to the energy consumption coefficient and service quality coefficient.

[0081] Specifically, the above step S202 includes:

[0082] Step S2021, based on the status information, obtain the network bandwidth, the memory occupied by the virtual machine, the maximum energy consumption, the idle probability, and the CPU utilization and memory utilization when the virtual machine is running.

[0083] Specifically, network bandwidth refers to the amount of data that can be transmitted in the network per unit time. The bandwidth usage of the port can be viewed through network management devices such as the router management interface. The memory occupied by the virtual machine refers to the size of the physical memory actually used during the operation of the virtual machine. The memory occupied by the virtual machine and the parameters such as the maximum energy consumption, idle probability, CPU utilization and memory utilization of the physical machine can all be obtained through the status information obtained by the cloud computing center from monitoring the physical and virtual machines.

[0084] Step S2022, calculating the physical machine power according to the maximum energy consumption, the idle probability, the CPU utilization and the memory utilization, and determining the energy consumption coefficient of the virtual machine when running according to the physical machine power.

[0085] Specifically, for the energy consumption problem of the cloud computing center, a reasonable virtual machine scheduling method can save the cloud computing center's resource expenditure, reduce the emission of waste gases such as carbon dioxide, reduce environmental treatment costs, and at the same time increase the life of the physical machine. The operation process of the virtual machine depends on physical machine resources such as CPU, memory, etc. The energy consumption of the physical machine is closely related to whether it is in an idle state and the CPU utilization and memory utilization rate during dynamic operation. Suppose the maximum energy consumption of a physical machine is W, and the energy consumption in the idle state is α (0≤α≤1) times that of the running state. Suppose the CPU utilization of the CPU processing task queue T at time t is URc(T,t), and the memory utilization of the task queue T at time t is URm(T,t). Suppose the idle probability of the physical machine is β. Therefore, the physical machine power W(T,t) at a certain time t is:

[0086]

[0087] Then, the energy consumption coefficient of the cloud data center from time t0 to time t1 when the virtual machine is running is:

[0088]

[0089] Step S2023, calculate the virtual machine jam coefficient according to the central processing unit utilization and the memory utilization, and calculate the virtual machine migration time according to the network bandwidth and the occupied memory, and determine the service quality coefficient of the virtual machine when it is running according to the virtual machine jam coefficient and the virtual machine migration time.

[0090] Specifically, in addition to the energy consumption problem of the cloud data center, customer needs must also be considered during the virtual machine scheduling process. The ultimate goal of virtual machine technology is to serve customers. Therefore, the present invention considers the impact on customer service quality from three aspects: CPU, memory, and bandwidth. First, from the perspective of the CPU, when the CPU utilization and memory utilization increase, the virtual machine will gradually become stuck, and the higher the CPU and memory utilization, the more obvious the stuck phenomenon. Therefore, this stuck phenomenon is expressed by the following formula:

[0091]

[0092] Among them, d(T, t) represents the impact of a single virtual machine's jamming on the service coefficient. An exponential function is used to reflect the negative correlation between utilization and virtual machine jamming. That is, the higher the CPU utilization and memory utilization, the greater the negative impact on the service coefficient. D is the jamming coefficient, which represents the impact of the average CPU delay and memory delay of each virtual machine on the service coefficient. URc(T, t) represents the CPU utilization of task queue T at time t, URm(T, t) represents the memory utilization of task queue T at time t, t1 represents the virtual machine shutdown time, and t0 represents the virtual machine startup time.

[0093] Secondly, when the performance of the virtual machine is not enough to meet the demand, there is a delay problem or a physical machine failure, etc., the virtual machine needs to be migrated online. The virtual machine migration time is:

[0094]

[0095] Among them, MT i Represents the virtual machine migration time, M i Represents the memory occupied by the virtual machine, B i Represents the network bandwidth, and i represents the virtual machine number. At the same time, due to network transmission delays or insufficient resources on the target host, virtual machine migration may cause the performance of the virtual machine to degrade. Therefore, the total virtual machine performance degradation estimation formula is as follows:

[0096]

[0097] Among them, p i (t) is the performance degradation caused by virtual machine migration. The coefficient of performance degradation can be obtained based on monitoring tools or log records. P is the impact of the performance degradation of the entire process on customer service.

[0098] Finally, by measuring the impact of the jamming according to the VM jamming coefficient and considering the performance degradation caused by the VM migration time, the service quality coefficient S(T, t0, t1) can be expressed by the following formula:

[0099] S(T,t0,t1)=50*(D+(1-P))

[0100] Step S2024, obtaining the virtual machine scheduling task target according to the energy consumption coefficient and the service quality coefficient.

[0101] Specifically, the task goal is to reduce energy consumption and improve service quality. According to the above energy consumption coefficient and service quality coefficient formula, the comprehensive coefficient Obj can be obtained:

[0102] Obj=S*E

[0103] Then, the goal of the virtual machine scheduling task is to find the maximum value of the comprehensive coefficient Obj in order to find a balance between low energy consumption and high service quality, thereby maximizing resource utilization and improving virtual machine performance.

[0104] During the virtual machine scheduling process, the present invention determines the energy consumption of the virtual machine during operation through the utilization rate of resources such as the central processing unit and memory on the one hand; on the other hand, uses the virtual machine jam coefficient to measure the virtual machine jam phenomenon caused by resource utilization, and considers the impact of migration on performance when the virtual machine jams and fails. The virtual machine jam coefficient and virtual machine migration time are used to measure the service quality of the virtual machine, reasonably utilize dynamically changing data, consider energy consumption and service coefficient issues, and optimize the virtual machine scheduling process.

[0105] Step S203: construct a decision space according to the state information and the virtual machine scheduling task target.

[0106] Specifically, the above step S203 includes:

[0107] Step S2031, constructing a state set according to the remaining resource quantities, current task requirements and current task total quantities corresponding to various virtual machine resources in the state information.

[0108] Specifically, virtual machine resources include at least some of the following items: central processing unit, memory and bandwidth. The state set ST represents a set of environmental information, which is crucial for subsequent decision-making. The present invention defines the state as a vector, which includes the number of various resources in the current system, the number of virtual machines of different types, and the amount of resources consumed by each type of virtual machine. The state set ST is defined as:

[0109]

[0110] in, represents the state of the i-th virtual machine at time t, They respectively represent the remaining CPU resources, the current task demand, and the current task total; They respectively represent the remaining memory resources, the current task demand, and the current task total; They respectively represent the remaining bandwidth resources, the current task demand and the current task total amount.

[0111] Furthermore, different virtual machines will have different states when executing different scheduling tasks at different times. After an action is completed, the virtual machine state will have ST t Change to ST t+1 .

[0112] Step S2032, constructing an action set according to the preset execution instructions.

[0113] Specifically, actions can be constructed according to various preset execution instructions, and the preset execution instructions include at least some of the following items: starting a virtual machine, shutting down a virtual machine, migrating a virtual machine, adding a virtual machine, reducing a virtual machine, and not executing an action. The action set A represents instructions executed on a virtual machine, and the action set A in any time period mainly includes various actions at, where at mainly includes six types of operations: starting a virtual machine, shutting down a virtual machine, migrating a virtual machine, adding a virtual machine, reducing a virtual machine, and not executing an action.

[0114] Step S2033, combining the state set and the action set to obtain multiple state transition combinations, and calculating the reward coefficient corresponding to each state transition combination according to the virtual machine scheduling task target.

[0115] Specifically, the state transition combination includes the current state of the virtual machine, the current action, and the updated state after executing the current action. The reward is the state After executing the action, at becomes the state The feedback obtained from the system, the feedback G, is the accumulation of all rewards over time. The present invention defines the reward set R as:

[0116]

[0117] in, is the reward coefficient, the reward coefficient It is set according to the energy consumption coefficient and the service quality coefficient (or the virtual machine scheduling task target). The virtual machine scheduling task target can better construct the reward function used in reinforcement learning, so as to reasonably use the dynamically changing data in the reinforcement learning process and consider the energy consumption and service coefficient issues more in the training process. The present invention defines the reward coefficient as:

[0118]

[0119] Among them, S(T j ,t,t+1) is the service quality coefficient, E(T j ,t,t+1) is the energy consumption coefficient. In addition, considering that adding and reducing virtual machines will increase the system burden and increase system overhead, the present invention will increase the reward coefficient Updated to:

[0120]

[0121] Among them, ||a t || is the penalty action a t paradigm, w is the weight coefficient for punishing the corresponding action.

[0122] Step S2034, constructing a decision space for virtual machine scheduling based on the state set, action set, state transition combination and corresponding reward coefficient.

[0123] Specifically, the state transition strategy PO is determined according to the state transition combination, and the decision space for virtual machine scheduling is constructed according to the state set ST, action set A, reward set R and feedback G constructed above. The decision space can be a five-tuple corresponding to the Markov decision process (MDP). It should be noted that the Markov decision process refers to a random process in which the conditional probability distribution of the future state depends only on the current state and all past states, and has nothing to do with the past state. This property can be simply referred to as "the future depends on the present, not on the past".

[0124] Specifically, in the virtual machine scheduling environment, the Markov process can be understood as an independent decision maker responsible for making scheduling decisions in a given virtual machine environment. The entire resource scheduling task is modeled as a Markov decision process, and the five-tuple representation formula of the Markov process (MDP) decision space is established:

[0125] MDP={ST,A,PO,R,G}

[0126] Among them, ST represents the state set, which indicates all possible states in the virtual machine scheduling environment; A represents the action set, which indicates all possible scheduling actions in the virtual machine scheduling environment; PO represents the strategy adopted for state transfer, which can be regarded as a mapping function from state to action in the virtual machine scheduling environment; R represents the reward set, which can be expressed as the feedback of the environment after the scheduling action is executed in the virtual machine scheduling environment; G is feedback, which is the accumulation of rewards over time.

[0127] The present invention utilizes the remaining resource quantity corresponding to the virtual machine resources, the current task demand quantity and the current task total quantity to determine the environment state of the virtual machine to construct a state set, and constructs an action set according to the instructions that the virtual machine may execute. Then, the state and action are combined, and the reward feedback after the virtual machine executes the action is calculated to determine the state transition strategy, reward set and feedback. Finally, according to the five-tuple consisting of the state set, action set, state transition strategy, reward set and feedback, the decision space for virtual machine scheduling is constructed to facilitate the determination of the virtual machine scheduling strategy.

[0128] Step S204, obtaining a task request sorting queue according to the task request queue and the trained task sorting model.

[0129] Specifically, the above step S204 includes:

[0130] Step S2041, determining the first attribute parameter of each user task request in the task request queue.

[0131] Specifically, the first attribute parameter includes at least part of the following items: task priority, arrival time, processing time, and task type.

[0132] Step S2042, using the first attribute parameter and the trained task sorting model, sort the execution order of each user task request in the task request queue to obtain a task request sorting queue.

[0133] Specifically, the task request queue is input into the trained lightweight task sorting model to obtain the execution order of each user's task request, sort the task request queue, and construct the task request sorting queue.

[0134] The present invention utilizes a trained task sorting model and combines multiple global data such as task priority, arrival time, processing time, task type, etc. of user task requests to sort the task request queue, thereby solving the problem of single consideration when sorting request tasks in traditional methods. By more reasonably sorting tasks before virtual machine scheduling, the efficiency of virtual machine scheduling is improved.

[0135] In some optional implementations, a task data set is first constructed, and a label is added to each scheduled task in the task data set. The second attribute parameter of the scheduled task in the task data set includes task priority, arrival time, processing time, waiting time from queuing to completion, and task type. The label of the scheduled task is obtained by weighted calculation based on the waiting time and task priority. Then, a first network model is constructed based on the first preset model structure, and the first network model is trained using the task data set to obtain a second network model. The output result of the second network model is the execution order and task completion time of the scheduled tasks. Finally, the predicted label of the scheduled task is calculated based on the execution order, task completion time, and task priority, and the first loss value of the second network model is calculated based on the label and the predicted label, and the model parameters of the second network model are adjusted based on the first loss value until the model converges to obtain a trained task sorting model.

[0136] Exemplarily, the first preset model structure can be a deep learning model such as a neural network model. Deep learning uses deep neural networks to simulate the human learning process. Through a large amount of data and calculations, the model can automatically extract and abstract useful features to complete various complex tasks.

[0137] In some optional embodiments, such as Figure 3 As shown in Figure 1, the training process of the task sorting model mainly includes the following steps:

[0138] Step a1, construct a lightweight request task sorting model (referred to as task sorting model) data set. This data set is mainly used to train the task sorting model. Traditional task sorting methods are often based on simple rules and strategies for scheduling, such as first-come-first-served, time slice round-robin algorithm, etc., which are not comprehensive. The sorting of the request queue will affect the efficiency of the entire virtual machine scheduling to a certain extent. Therefore, the present invention will use a lightweight deep learning model to perform virtual request queue sorting and improve the efficiency of virtual machine scheduling.

[0139] Specifically, a data set containing various scheduling tasks is collected. These data come from the task scheduling results generated by the simulation tasks. Each scheduling task contains the following attributes as input parameters: task priority, arrival time, processing time, and task type. The collected raw data is cleaned, converted, and standardized, and outliers are filled with medians and null values ​​are filled with 0 to facilitate model learning and prediction. Then, labels are added to the virtual machine scheduling tasks. The labels are weighted by the time from queuing to completion of each scheduling task and the task priority is used. The formula is as follows:

[0140] l i =t i *p i

[0141] Among them, l i is the label of the i-th scheduled task, t i is the time from queuing to completion, p i is the weight corresponding to the task priority.

[0142] Furthermore, the labels are normalized. It should be noted that the labels of the dataset need to be manually sorted by relevant experts and calculated after multiple task simulations. In order to verify the performance of the model, the dataset is divided into a training set and a test set.

[0143] Step a2: training the task sorting model. The specific process is as follows:

[0144] Step a21, set model hyperparameters such as learning rate δ, training cycle epoch, and batch data size batchsize.

[0145] Step a22, using smaller convolution kernels and fewer channels to reduce the number of model parameters, to build a lightweight neural network model, and send the data set to the lightweight neural network model for supervised learning training. The output result is the execution order of the task and the task completion time.

[0146] Step a23, calculate the loss value, calculate the task prediction label according to the task execution order, task completion time and task priority, calculate the loss value according to the prediction label, and adjust the model parameters according to the loss value.

[0147] Step a24, using the EarlyStopping method to determine whether the model has converged, if not, repeat steps a22 to a23; if converged, stop training to obtain a trained task sorting model.

[0148] The present invention constructs a data set using global data such as task priority, arrival time, processing time, waiting time from queuing to completion, and task type of the scheduled tasks, and trains a task sorting model, so that the trained task sorting model can consider multiple factors when sorting user task requests, thereby improving the rationality of task sorting, reducing resource waste caused by unreasonable sorting, facilitating virtual machine scheduling to process tasks in a reasonable order, and improving user experience.

[0149] Step S205, using the task request sorting queue, the decision space and the trained virtual machine resource scheduling model to obtain the virtual machine scheduling strategy.

[0150] Specifically, the above step S205 includes:

[0151] Step S2051, input the task request sorting queue and the decision space into the trained virtual machine resource scheduling model, predict the reward value of the virtual machine executing the corresponding action in different states, and obtain the reward prediction result.

[0152] Specifically, the decision space is a five-tuple including a state set, an action set, a state transition, a reward set, and a return set. The user request process and virtual machine resources are simulated using a simulation environment to obtain the simulated task request queue and the simulated state information of the virtual machine resources. Next, the simulated task request queue is sorted based on the trained task sorting model to obtain the simulated task request sorting queue, and an experience pool is constructed based on the simulated state information, wherein the experience pool includes the reward value when the virtual machine performs the corresponding action in different states. Then, a first scheduling model is constructed based on the second preset model structure, and the first scheduling model is trained using the simulated task request sorting queue and the experience pool to obtain a second scheduling model, wherein the output result of the second scheduling model is the maximum reward prediction value when the virtual machine performs the corresponding action in different states. Finally, the second loss value of the second scheduling model is calculated, and the model parameters of the second scheduling model are adjusted based on the second loss value until the model converges to obtain a trained virtual machine resource scheduling model.

[0153] In some optional embodiments, such as Figure 4 As shown, the training process of the virtual machine resource scheduling model includes the following steps:

[0154] Step b1, build a simulation environment. CloudSim simulation software can be used to build a simulation experiment of a cloud computing environment. CloudSim is a powerful simulation tool that can realistically simulate the virtual machine scheduling process and provides an interface for interacting with the Deep Q-Network (DQN) model, which can be used to train and test reinforcement learning algorithms.

[0155] It should be noted that the DQN model is an algorithm that combines deep learning and reinforcement learning. It estimates the Q value by building a deep neural network, thereby solving the Q-Learning problem in reinforcement learning. Reinforcement learning is a type of deep learning that learns by interacting with the environment, and guides the learning process through strategy iteration and reward functions, thereby maximizing cumulative rewards and achieving intelligent decision-making.

[0156] Specifically, in the simulation environment constructed above, multiple data centers can be created to simulate the role of a cloud service provider. Each data center has its own unique resources and performance characteristics, such as processors, memory, storage, etc., which reflect the characteristics of an actual data center. At the same time, in order to simulate the arrival process of user requests more realistically, a separate data center is created to simulate the arrival of user requests. This data center simulates the arrival model of user requests, such as Poisson distribution, exponential distribution, etc., which reflect the arrival pattern of actual user requests.

[0157] Furthermore, in terms of setting the type and number of virtual machines, they are set according to the actual situation. Different virtual machine types have different processor, memory and storage requirements, reflecting the diversity of the actual cloud computing environment. At the same time, the number of virtual machines in each data center can be set according to demand to reflect the load conditions in actual operation. The types and numbers of these virtual machines are recorded in the state set ST for training and testing of the DQN model.

[0158] Step b2: training a virtual machine resource scheduling model based on DQN.

[0159] Specifically, the DQN model is a model-free reinforcement learning algorithm based on a value function. It uses a neural network to obtain the Q value of the state and action. The DQN model is improved on the basis of Q-Learning, and can more effectively handle high-dimensional state and action space, while having better generalization performance. The core of the DQN algorithm is to use a deep neural network to approximate the state-behavior mapping, so as to make decisions without a model. The algorithm uses a neural network to learn the Q function. The input of the network is the current state, and the output is the Q value of each possible action. By constantly interacting with the environment, the DQN algorithm can gradually adjust the parameters of the neural network to obtain a more accurate Q value estimate. The training process is as follows:

[0160] Step b21, initialize the CloudSim simulation library, create a data center and a user request center, and set the number and type of physical machines contained in the data center. The user request center is also responsible for connecting the main parameters of the cloud computing user task request and the computing service task request provided by the cloud service provider, such as the task computing amount, task completion time, etc. We use Poisson distribution to generate the parameters, and the arrival time of the task request satisfies the exponential distribution. The user request center will match the cloud computing user request based on the information it has.

[0161] Step b22: pre-process the task requests using a lightweight task sorting model and pre-sort the task requests.

[0162] Step b23, set the reinforcement learning parameters, set the learning rate δ, the target Q network update step size c, the initialization parameter Φ, the target Q network parameter Φ ， =Φ, discount rate γ, simulated state set ST and action set A, initialize the experience pool D, set the capacity to N, and select a suitable loss function.

[0163] Step b24, determine the initialization state s.

[0164] Step b25, randomly select an action at.

[0165] Step b26, execute action at and obtain the next state st, and calculate the reward value r.

[0166] Step b27, store the four-tuple (s, at, r, St) into the experience pool.

[0167] Step b28: Every time the step length c is reached, the parameters of the evaluation network are assigned to the target Q network, samples are extracted from the experience pool, and the network is trained.

[0168] Step b29, use the target network to calculate the maximum Q value in the next state, calculate the loss value, and back-propagate to update the network parameters.

[0169] Step b210, when the state is the terminal state, a segment learning is completed, and steps b25 to b210 are repeated.

[0170] Step b211, the network converges, the execution ends, and the Q-Learning table and the scheduling policy Policy are obtained. The Q-Learning table is used to store the state-action corresponding Q value table.

[0171] The present invention utilizes a simulation environment to simulate the arrival process of user requests and various virtual machine resources, and then trains a scheduling model based on the simulation data, so that the trained virtual machine resource scheduling model can calculate the reward value after executing an action based on a state set and an action set, thereby determining the scheduling strategy for the virtual machine according to the maximum reward value.

[0172] Step S2052, obtaining a virtual machine scheduling strategy corresponding to the task request sorting queue according to the reward prediction result.

[0173] Specifically, the reward value after executing each action in the current state can be obtained according to the reward prediction result, and the scheduling strategy for the user task request can be determined according to the target action corresponding to the maximum reward value.

[0174] like Figure 5 As shown, the present invention proposes a virtual machine scheduling method of a static and dynamic dual lightweight model, aiming to solve the above-mentioned problems existing in the current virtual machine scheduling method. The present invention proposes a two-stage model combining dynamic and static. In the first stage, a globally related static data request task sorting model is used to preprocess the request queue. In the second stage, a reward mechanism is constructed based on the virtual machine scheduling task goal proposed by the present invention, and a reinforcement learning algorithm is applied to perform dynamic resource scheduling, so that the dynamic scheduling method starts from dynamic data changes more comprehensively, and considers energy consumption and service coefficient issues. The present invention considers the virtual machine scheduling problem from both static and dynamic aspects, reasonably utilizes the data generated before and during virtual machine scheduling, and more effectively combines global planning and dynamic changes, thereby improving resource utilization, improving system performance, and realizing a better virtual machine scheduling method.

[0175] See again Figure 5 In actual applications, after obtaining user task requests, the task request queue is sent to the lightweight request task sorting model for sorting. Each subsequent request task will be sorted again. After sorting, the task request sorting queue is obtained. The scheduler processes tasks according to the request queue. The Q-Learning table and scheduling strategy are obtained based on the trained virtual machine resource scheduling model and applied to the production environment for virtual machine scheduling. At the same time, in order to make the model better adapt to the production environment, the trained Q-Learning table is used as the initial Q-Learning of the dynamic virtual machine scheduling algorithm, and the reward function constructed based on the virtual machine scheduling task target is used in the DQN algorithm and deployed to the production environment to train a policy that is more suitable for the production environment. A sufficiently large update step is set to ensure that the model can be updated regularly within a certain period of time, thereby achieving virtual machine scheduling that is more in line with the production environment.

[0176] Compared with the existing virtual machine scheduling method, the present invention proposes a virtual machine scheduling algorithm of static and dynamic dual lightweight models, which no longer mechanically arranges and combines data for model training, but classifies data for training models, makes better use of data, and more effectively combines global planning and dynamic changes, thereby improving resource utilization, improving system performance, and achieving a better virtual machine scheduling method. At the same time, the two lightweight models reasonably utilize the global static historical data before virtual machine scheduling and the dynamically changing data generated during virtual machine scheduling, which have better effects in terms of data utilization and reducing computing pressure compared to mechanically feeding data into a complex model.

[0177] In terms of the first-stage static request task sorting model, the present invention utilizes a variety of global data such as task priority, arrival time, processing time, task type, etc. to train a lightweight request task sorting model, which solves the problem of traditional methods sorting request tasks based on a single factor. By more reasonably sorting tasks before scheduling virtual machines, the efficiency of virtual machine scheduling is improved.

[0178] In terms of the second-stage dynamic virtual machine scheduling model, a reward mechanism is constructed based on the virtual machine scheduling task objectives proposed in the present invention. The reward mechanism designs energy consumption formulas and service coefficient formulas from the aspects of dynamic virtual machine CPU utilization, memory utilization, and virtual machine scheduling operations. The reward mechanism is applied to reinforcement learning, so that reinforcement learning comprehensively considers energy consumption and service systems from multiple aspects, allowing the entire virtual machine scheduling algorithm to be more in line with the requirements of modern society for energy consumption and the company's emphasis on customer value.

[0179] In this embodiment, a virtual machine scheduling device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0180] This embodiment provides a virtual machine scheduling device, such as Figure 6 As shown, including:

[0181] The acquisition module 601 is used to acquire user task requests and construct a task request queue, and acquire status information of virtual machine resources;

[0182] The first processing module 602 is used to determine the energy consumption coefficient and the service quality coefficient of the virtual machine when it is running based on the state information, and obtain the virtual machine scheduling task target according to the energy consumption coefficient and the service quality coefficient;

[0183] The second processing module 603 is used to construct a decision space according to the state information and the virtual machine scheduling task target;

[0184] The third processing module 604 is used to obtain a task request sorting queue according to the task request queue and the trained task sorting model;

[0185] The fourth processing module 605 is used to obtain a virtual machine scheduling strategy by using the task request sorting queue, the decision space and the trained virtual machine resource scheduling model.

[0186] In some optional implementations, the first processing module 602 is further configured to:

[0187] Based on the status information, the network bandwidth, the virtual machine's memory usage, the maximum energy consumption, the idle probability, and the CPU utilization and memory utilization of the virtual machine when it is running are obtained;

[0188] The physical machine power is calculated based on the maximum energy consumption, idle probability, CPU utilization, and memory utilization, and the energy consumption coefficient of the virtual machine when running is determined based on the physical machine power;

[0189] The virtual machine jam coefficient is calculated based on the CPU utilization and memory utilization, and the virtual machine migration time is calculated based on the network bandwidth and occupied memory. The service quality coefficient of the virtual machine during operation is determined based on the virtual machine jam coefficient and the virtual machine migration time.

[0190] In some optional implementations, the second processing module 603 is further configured to:

[0191] Constructing a state set according to the remaining resource quantity, current task demand quantity and current task total quantity corresponding to various virtual machine resources in the state information; wherein the virtual machine resources include at least some of the following items: central processing unit, memory and bandwidth;

[0192] Constructing an action set according to a preset execution instruction; wherein the preset execution instruction includes at least some of the following items: starting a virtual machine, shutting down a virtual machine, migrating a virtual machine, adding a virtual machine, reducing a virtual machine, and not executing an action;

[0193] Combine the state set and the action set to obtain multiple state transition combinations, and calculate the reward coefficient corresponding to each state transition combination according to the virtual machine scheduling task target; wherein the state transition combination includes the current state of the virtual machine, the current action, and the updated state after executing the current action;

[0194] According to the state set, action set, state transition combination and corresponding reward coefficient, the decision space for virtual machine scheduling is constructed.

[0195] In some optional embodiments, the device is also used for:

[0196] Construct a task data set and add a label to each scheduled task in the task data set; the second attribute parameter of the scheduled task includes task priority, arrival time, processing time, waiting time from queuing to completion, and task type. The label is obtained by weighted calculation based on the waiting time and task priority.

[0197] A first network model is constructed based on the first preset model structure, and the first network model is trained using the task data set to obtain a second network model; wherein the output result of the second network model is the execution order and task completion time of the scheduled tasks;

[0198] The predicted label of the scheduled task is calculated according to the execution order, task completion time and task priority, and the first loss value of the second network model is calculated according to the label and the predicted label, and the model parameters of the second network model are adjusted based on the first loss value until the model converges to obtain a trained task sorting model.

[0199] In some optional implementations, the third processing module 604 is further configured to:

[0200] Determine a first attribute parameter of each user task request in the task request queue; wherein the first attribute parameter includes at least part of the following items: task priority, arrival time, processing time and task type;

[0201] The first attribute parameter and the trained task sorting model are used to sort the execution order of each user task request in the task request queue to obtain a task request sorting queue.

[0202] In some optional embodiments, the device is also used for:

[0203] The user request process and virtual machine resources are simulated using the simulation environment to obtain simulation status information of the simulation task request queue and virtual machine resources;

[0204] The simulated task request queue is sorted based on the trained task sorting model to obtain the simulated task request sorting queue, and an experience pool is constructed according to the simulation state information; wherein the experience pool includes the reward value when the virtual machine performs the corresponding action in different states;

[0205] A first scheduling model is constructed based on a second preset model structure, and the first scheduling model is trained using a simulated task request sorting queue and an experience pool to obtain a second scheduling model; wherein an output result of the second scheduling model is a maximum reward prediction value when the virtual machine performs corresponding actions in different states;

[0206] A second loss value of the second scheduling model is calculated, and model parameters of the second scheduling model are adjusted based on the second loss value until the model converges, thereby obtaining a trained virtual machine resource scheduling model.

[0207] In some optional implementations, the fourth processing module 605 is further configured to:

[0208] Input the task request sorting queue and decision space into the trained virtual machine resource scheduling model, predict the reward value of the virtual machine executing the corresponding action in different states, and obtain the reward prediction result; wherein the decision space includes the state set, action set, state transition, reward set and return set;

[0209] According to the reward prediction results, the virtual machine scheduling strategy corresponding to the task request sorting queue is obtained.

[0210] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0211] The virtual machine scheduling device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0212] The embodiment of the present invention also provides a computer device having the above Figure 6 The virtual machine scheduling device is shown.

[0213] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0214] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0215] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0216] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0217] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0218] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0219] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0220] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0221] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A virtual machine scheduling method, characterized in that: The method comprises: Obtain user task requests and construct a task request queue, as well as obtain status information of virtual machine resources; Determine the energy consumption coefficient and service quality coefficient of the virtual machine when it is running based on the state information, and obtain the virtual machine scheduling task target according to the energy consumption coefficient and service quality coefficient; Construct a decision space based on the state information and the virtual machine scheduling task objectives; According to the task request queue and the trained task sorting model, a task request sorting queue is obtained; The virtual machine scheduling strategy is obtained by using the task request sorting queue, decision space and the trained virtual machine resource scheduling model.

2. The method according to claim 1, characterized in that The step of determining the energy consumption coefficient and the service quality coefficient of the virtual machine when it is running based on the state information includes: Based on the status information, the network bandwidth, the virtual machine's memory usage, the maximum energy consumption, the idle probability, and the CPU utilization and memory utilization of the virtual machine when it is running are obtained; The physical machine power is calculated based on the maximum energy consumption, idle probability, CPU utilization, and memory utilization, and the energy consumption coefficient of the virtual machine when running is determined based on the physical machine power; The virtual machine jam coefficient is calculated based on the CPU utilization and memory utilization, and the virtual machine migration time is calculated based on the network bandwidth and occupied memory. The service quality coefficient of the virtual machine during operation is determined based on the virtual machine jam coefficient and the virtual machine migration time.

3. The method according to claim 2, characterized in that The step of constructing a decision space according to the state information and the virtual machine scheduling task target includes: Constructing a state set according to the remaining resource quantity, current task demand quantity and current task total quantity corresponding to various virtual machine resources in the state information; wherein the virtual machine resources include at least some of the following items: central processing unit, memory and bandwidth; Constructing an action set according to a preset execution instruction; wherein the preset execution instruction includes at least some of the following items: starting a virtual machine, shutting down a virtual machine, migrating a virtual machine, adding a virtual machine, reducing a virtual machine, and not executing an action; The state set and the action set are combined to obtain multiple state transition combinations, and the reward coefficient corresponding to each state transition combination is calculated according to the virtual machine scheduling task target; wherein the state transition combination includes the current state of the virtual machine, the current action, and the updated state after executing the current action; According to the state set, action set, state transition combination and corresponding reward coefficient, the decision space for virtual machine scheduling is constructed.

4. The method according to any one of claims 1 to 3, characterized in that The step of obtaining the task request sorting queue according to the task request queue and the trained task sorting model includes: Determine a first attribute parameter of each user task request in the task request queue; wherein the first attribute parameter includes at least part of the following items: task priority, arrival time, processing time and task type; The first attribute parameter and the trained task sorting model are used to sort the execution order of each user task request in the task request queue to obtain a task request sorting queue.

5. The method according to claim 4, characterized in that The training process of the task sorting model includes: Construct a task data set and add a label to each scheduled task in the task data set; the second attribute parameter of the scheduled task includes task priority, arrival time, processing time, waiting time from queuing to completion, and task type. The label is obtained by weighted calculation based on the waiting time and task priority. A first network model is constructed based on the first preset model structure, and the first network model is trained using the task data set to obtain a second network model; wherein the output result of the second network model is the execution order and task completion time of the scheduled tasks; The predicted label of the scheduled task is calculated according to the execution order, task completion time and task priority, and the first loss value of the second network model is calculated according to the label and the predicted label, and the model parameters of the second network model are adjusted based on the first loss value until the model converges to obtain a trained task sorting model.

6. The method according to any one of claims 1 to 3, characterized in that The virtual machine scheduling strategy is obtained by utilizing the task request sorting queue, the decision space and the trained virtual machine resource scheduling model, including: Input the task request sorting queue and decision space into the trained virtual machine resource scheduling model, predict the reward value of the virtual machine executing the corresponding action in different states, and obtain the reward prediction result; wherein the decision space includes the state set, action set, state transition, reward set and return set; According to the reward prediction results, the virtual machine scheduling strategy corresponding to the task request sorting queue is obtained.

7. The method according to claim 6, characterized in that The training process of the virtual machine resource scheduling model includes: The user request process and virtual machine resources are simulated using the simulation environment to obtain simulation status information of the simulation task request queue and virtual machine resources; The simulated task request queue is sorted based on the trained task sorting model to obtain the simulated task request sorting queue, and an experience pool is constructed according to the simulation state information; wherein the experience pool includes the reward value when the virtual machine performs the corresponding action in different states; A first scheduling model is constructed based on a second preset model structure, and the first scheduling model is trained using a simulated task request sorting queue and an experience pool to obtain a second scheduling model; wherein an output result of the second scheduling model is a maximum reward prediction value when the virtual machine performs corresponding actions in different states; A second loss value of the second scheduling model is calculated, and model parameters of the second scheduling model are adjusted based on the second loss value until the model converges, thereby obtaining a trained virtual machine resource scheduling model.

8. A virtual machine scheduling device, characterized in that: The device comprises: An acquisition module is used to acquire user task requests and construct a task request queue, as well as to acquire status information of virtual machine resources; The first processing module is used to determine the energy consumption coefficient and the service quality coefficient of the virtual machine when it is running based on the state information, and obtain the virtual machine scheduling task target according to the energy consumption coefficient and the service quality coefficient; The second processing module is used to construct a decision space according to the state information and the virtual machine scheduling task target; The third processing module is used to obtain a task request sorting queue according to the task request queue and the trained task sorting model; The fourth processing module is used to obtain a virtual machine scheduling strategy by using the task request sorting queue, the decision space and the trained virtual machine resource scheduling model.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the virtual machine scheduling method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the virtual machine scheduling method according to any one of claims 1 to 7.

Images