Virtual machine resource scheduling method and device, equipment and storage medium

Abstract

The invention discloses a virtual machine resource scheduling method, device and equipment and a storage medium, and relates to the technical field of cloud computing, and the method comprises the steps: obtaining a to-be-scheduled task set of business software; wherein the number of to-be-scheduled tasks in the to-be-scheduled task set is greater than the number of virtual machines in the virtual machine set; determining to-be-scheduled tasks corresponding to the virtual machines in the virtual machine set by using a double-Q learning algorithm; wherein each to-be-scheduled task in the to-be-scheduled task set corresponds to one virtual machine; all to-be-scheduled tasks in the to-be-scheduled task set are distributed to the corresponding virtual machines respectively; according to the method, a more stable and accurate scheduling strategy can be given in a dynamically changing workload and resource competition scene in a cloud computing environment by utilizing the good representation capability of a double-Q learning algorithm, and the adaptability of a system in a complex dynamic environment is enhanced; and the average completion time of the business task can be reduced, and the user service quality requirement is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, and particularly relates to a virtual machine resource scheduling method and device, equipment and a storage medium. BACKGROUND

[0002] Cloud computing, as a powerful computing model, is widely used in various fields, providing more efficient, flexible and innovative solutions for organizations and individuals. With the increasing demand for cloud computing, users' requirements for the quality of cloud services are also increasing. In the cloud computing environment, virtual machine resource scheduling refers to allocating virtual machine resources to simultaneously arriving tasks and determining the order of virtual machine resource scheduling for complex resource requirements of multiple users, such as the combined requirements of heterogeneous central processing units (CPUs), graphics processing units (GPUs), memory and storage resources. Therefore, the primary goal of virtual machine resource scheduling is to achieve efficient utilization of resources, reduce user waiting time and ensure the quality of service of real-time tasks.

[0003] In related technologies, many classic resource scheduling algorithms are applied to the allocation of virtual machine resources, such as shortest job first, highest response ratio first and minimum remaining time first algorithms; these algorithms mainly make scheduling decisions based on factors such as task priority, execution time or resource requirements. However, these algorithms have certain limitations in dealing with dynamic changes in workloads and resource competition in the cloud computing environment. Therefore, how to provide a virtual machine resource scheduling method suitable for the cloud computing environment, reduce user waiting time and meet user service quality requirements is a problem that needs to be solved today. SUMMARY

[0004] The purpose of the present application is to provide a virtual machine resource scheduling method, device, equipment and computer readable storage medium to realize virtual machine resource scheduling suitable for the cloud computing environment, reduce user waiting time and meet user service quality requirements.

[0005] To solve the above technical problems, the present application provides a virtual machine resource scheduling method, comprising: obtaining a set of to-be-scheduled tasks of business software; wherein the number of to-be-scheduled tasks in the set of to-be-scheduled tasks is greater than the number of virtual machines in a virtual machine group, and the resource amount of each virtual machine meets the demand resource amount of any to-be-scheduled task; determining the respective to-be-scheduled tasks corresponding to each virtual machine in the virtual machine group by using a double Q learning algorithm; wherein each to-be-scheduled task in the set of to-be-scheduled tasks corresponds to a virtual machine, and the difference between the number of to-be-scheduled tasks corresponding to any two virtual machines is less than or equal to an equalization threshold; allocating all to-be-scheduled tasks in the set of to-be-scheduled tasks to their respective virtual machines respectively.

[0006] In another aspect, the double Q-learning algorithm is used to determine the respective virtual machine corresponding to each task to be scheduled in the virtual machine group, comprising: According to the preset state, the preset first Q-value function table and the preset second Q-value function table of the double Q-learning algorithm, a current task to be scheduled is determined from a current task set to be scheduled; wherein the current task to be scheduled is a target state-action pair corresponding to the task to be scheduled in the current task set to be scheduled, the target state-action pair is a state-action pair corresponding to the maximum average of the first Q-value and the second Q-value of each state-action pair corresponding to each task to be scheduled in the current task set to be scheduled; the preset first Q-value function table includes the first Q-value corresponding to each state-action pair, the preset second Q-value function table includes the second Q-value corresponding to each state-action pair, and the state-action pair includes the corresponding relationship between each task to be scheduled in the task set to be scheduled and each virtual machine in the virtual machine group; The virtual machine corresponding to the target state-action pair is determined as the virtual machine corresponding to the current task to be scheduled.

[0007] In another aspect, the method further comprises: Initializing a first Q-value function table, a second Q-value function table and a state space; wherein the state space is a matrix ; n is the number of training scheduling tasks in a preset scheduling task training set, m is the number of preset virtual machines in a preset virtual machine group; the value of d ij is 1 or 0, and d ij is 1 indicates that the i-th training scheduling task is assigned to the j-th virtual machine. According to the current state space, a current training scheduling task is selected by using a greedy strategy, and a preset virtual machine corresponding to the current training scheduling task is determined; wherein the current training scheduling task is any training scheduling task in the preset scheduling task training set; After the current training scheduling task is assigned to the corresponding preset virtual machine, an environmental feedback reward and a next state of the state space are obtained; wherein , r is the environmental feedback reward, C is the preset scheduling task training set, t c is the execution time of the training scheduling task c; According to the environmental feedback reward and the next state, the target Q-value function table is updated according to a preset update selection probability, to obtain an updated target Q-value function table; wherein the target Q-value function table is the first Q-value function table or the second Q-value function table; According to the next state, the state space is updated; determining whether the current iteration reaches a single-iteration termination condition, wherein the single-iteration termination condition comprises completion of the distribution of the preset scheduling task training set and / or the training step number of the current iteration reaching a step number threshold; if the single-iteration termination condition is reached, determining whether a training completion condition is met, wherein the training completion condition comprises the iteration number reaching an iteration number threshold and / or the total completion time of the training scheduling task corresponding to the current iteration being within a preset time range; if the single-iteration termination condition is not reached or the training completion condition is not met, performing the step of selecting the current training scheduling task according to the current state space and using the greedy strategy to determine the preset virtual machine corresponding to the current training scheduling task; if the training completion condition is met, determining the current task space as the preset state, the current first Q value function table as the preset first Q value function table, and the current second Q value function table as the preset second Q value function table.

[0008] In another aspect, the method further comprises: obtaining usage resource data of the business software corresponding to each preset computing resource index, wherein the preset computing resource index comprises at least one of central processing unit usage rate, memory occupancy, storage usage, and graphics processing unit usage rate; obtaining, according to the usage resource data, usage resource prediction data after a preset time step for each preset computing resource index using a summation autoregressive moving average model corresponding to each preset computing resource index; adjusting the virtual machines in the virtual machine group according to the usage resource prediction data.

[0009] In another aspect, the summation autoregressive moving average model corresponding to the current preset computing resource index is ; wherein the current preset computing resource index is any of the preset computing resource index, to is a trained autoregressive parameter, to is a trained moving average parameter, p is a preset autoregressive order, q is a preset moving average order, and T is the preset time step, is the usage resource prediction data of the current preset computing resource index after the preset time step, is the usage resource data of the current preset computing resource index at a time step before the preset time step after the current time t, is the prediction error of the current preset computing resource index at the q time steps before the preset time step after the current time t.

[0010] In another aspect, the method further comprises: obtaining model training data corresponding to each of the preset computing resource indicators; training, according to the model training data, an initial summation autoregressive moving average model corresponding to each of the preset computing resource indicators to obtain model parameters of the trained initial summation autoregressive moving average model, so as to construct a summation autoregressive moving average model corresponding to each of the preset computing resource indicators by using the model parameters.

[0011] In another aspect, the obtaining of the model training data corresponding to each of the preset computing resource indicators comprises: obtaining historical resource usage data of the business software corresponding to each of the preset computing resource indicators; preprocessing the historical resource usage data to obtain the model training data; wherein the preprocessing comprises at least one of missing value filling processing, outlier detection and correction processing, and data standardization processing.

[0012] In another aspect, the training, according to the model training data, of the initial summation autoregressive moving average model corresponding to each of the preset computing resource indicators to obtain the model parameters of the trained initial summation autoregressive moving average model comprises: dividing the model training data according to a preset division ratio to obtain a training set and a test set corresponding to each of the preset computing resource indicators; iteratively training, by using the training set, the initial summation autoregressive moving average model corresponding to each of the preset computing resource indicators, and adjusting target model parameters of the initial summation autoregressive moving average model by using a maximum likelihood estimation algorithm to obtain an iteratively trained initial summation autoregressive moving average model; verifying, by using the test set, the iteratively trained initial summation autoregressive moving average model corresponding to each of the preset computing resource indicators to obtain model parameters of an initial summation autoregressive moving average model that passes the verification.

[0013] In another aspect, the initial summation autoregressive moving average model corresponding to the current preset computing resource indicator is ; wherein, a is a preset error, B is a preset difference operator, and d is a preset difference order.

[0014] In another aspect, the method further comprises: determining, by using a grid search, the preset difference order, the preset autoregressive order, and the preset moving average order in a preset search range by using the Akaike information criterion or the Bayesian information criterion; wherein the preset search range is {0, 1, 2, 3}.

[0015] In another aspect, the initial sum autoregressive moving average model corresponding to each preset computing resource index is verified respectively by using the test set, and the model parameters of the initial sum autoregressive moving average model that passes the verification are obtained, including: According to the test set corresponding to the current preset computing resource index, the evaluation value of the initial sum autoregressive moving average model corresponding to the current preset computing resource index is determined by using , wherein MSE is the evaluation value, n is the number of samples in the test set corresponding to the current preset computing resource index, is the use resource prediction data of the ith sample, is the future real data of the use resource of the ith sample; If the evaluation value reaches the evaluation threshold, it is determined that the initial sum autoregressive moving average model corresponding to the current preset computing resource index passes the verification.

[0016] In another aspect, the virtual machines in the virtual machine group include deployed virtual machines and undeployed virtual machines, and the undeployed virtual machines are virtual machines that need to be expanded according to the use resource prediction data.

[0017] The application also provides a virtual machine resource scheduling device, including: A task acquisition module is configured to acquire a set of to-be-scheduled tasks of business software; wherein the number of to-be-scheduled tasks in the set of to-be-scheduled tasks is greater than the number of virtual machines in a virtual machine group, and the resource amount of each virtual machine meets the demand resource amount of any to-be-scheduled task; A task determination module is configured to determine the to-be-scheduled tasks corresponding to each virtual machine in the virtual machine group by using a double Q learning algorithm; wherein each to-be-scheduled task in the set of to-be-scheduled tasks corresponds to a virtual machine, and the difference between the number of to-be-scheduled tasks corresponding to any two virtual machines is less than or equal to an equalization threshold; A task allocation module is configured to allocate all to-be-scheduled tasks in the set of to-be-scheduled tasks to the corresponding virtual machines respectively.

[0018] The application also provides a virtual machine resource scheduling device, including: A memory is configured to store a computer program; A processor is configured to execute the computer program to realize the steps of the virtual machine resource scheduling method described above.

[0019] In addition, the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the virtual machine resource scheduling method described above.

[0020] The virtual machine resource scheduling method provided by the application comprises the following steps: obtaining a set of to-be-scheduled tasks of service software; wherein the number of to-be-scheduled tasks in the set of to-be-scheduled tasks is greater than the number of virtual machines in a virtual machine group, and the resource amount of each virtual machine meets the demand resource amount of any to-be-scheduled task; determining the respective to-be-scheduled tasks corresponding to each virtual machine in the virtual machine group by using a double Q learning algorithm; wherein each to-be-scheduled task in the set of to-be-scheduled tasks corresponds to a virtual machine, and the difference between the number of to-be-scheduled tasks corresponding to any two virtual machines is less than or equal to an equalization threshold; and distributing all to-be-scheduled tasks in the set of to-be-scheduled tasks to the respective virtual machines corresponding thereto respectively.

[0021] It can be seen that, by using the double Q learning algorithm to determine the respective to-be-scheduled tasks corresponding to each virtual machine in the virtual machine group, the application can use the good representation capability of the double Q learning algorithm to give more stable and accurate scheduling strategies under the dynamic changing work load and resource competition scene in the cloud computing environment, and enhance the adaptability of the system in the complex dynamic environment; and the average completion time of service tasks can be reduced, and the service quality requirement of users can be met. In addition, the application also provides a virtual machine resource scheduling device, equipment and computer readable storage medium, which also have the above beneficial effects. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute the embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0023] Figure 1 The flow chart of the virtual machine resource scheduling method provided by the embodiment of the application; Figure 2 The schematic diagram of the task allocation process provided by the embodiment of the application; Figure 3 The flow chart of the training process of the double Q learning algorithm provided by the embodiment of the application; Figure 4 The flow chart of the resource prediction adjustment process provided by the embodiment of the application; Figure 5 The schematic diagram of another framework of the virtual machine resource scheduling method provided by the embodiment of the application; Figure 6 The workflow schematic diagram of the resource prediction provided by the embodiment of the application; Figure 7A structural block diagram of a virtual machine resource scheduling device provided by an embodiment of the present application; Figure 8 A structural schematic diagram of a virtual machine resource scheduling device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0025] Please refer to Figure 1 , Figure 1 A flowchart of a virtual machine resource scheduling method provided by an embodiment of the present application. The method can include: Step 101: obtaining a set of to-be-scheduled tasks of a service software; wherein the number of to-be-scheduled tasks in the set of to-be-scheduled tasks is greater than the number of virtual machines in a virtual machine group, and the resource amount of each virtual machine satisfies the required resource amount of any to-be-scheduled task.

[0026] It can be understood that the service software in the embodiment can be an application software (such as a service application in the embodiment) which needs to call virtual machine resources for task processing in a service. Figure 2 The set of to-be-scheduled tasks in the embodiment can be a collection of all jobs (i.e. to-be-scheduled tasks) submitted to the service software at the same time, such as a collection of jobs recorded in a global job list (JobList).

[0027] Correspondingly, the to-be-scheduled tasks in the set of to-be-scheduled tasks in the embodiment can be job tasks which do not have a dependency relationship with each other, and the required resource amount of each to-be-scheduled task does not exceed the resource amount contained in any virtual machine in the virtual machine group, so that any virtual machine in the virtual machine group can process any to-be-scheduled task; for example, the to-be-scheduled tasks can be minimum granularity indivisible job tasks which have been divided.

[0028] Correspondingly, the virtual machine group in the embodiment can be a combination of virtual machines that are set in advance for processing tasks to be scheduled. The specific virtual machines in the virtual machine group can be set by the designer according to practical scenarios and user requirements, for example, each virtual machine can be a single core, so that the single-core virtual machine cannot utilize the excess CPU (Central Processing Unit) to execute other job tasks. That is, the virtual machine resources can adopt a time-sharing exclusive mechanism, that is, only one task to be scheduled can be executed on each virtual machine at any time. The task to be scheduled can be executed in a non-preemptive manner, that is, after the task to be scheduled enters the waiting queue of the virtual machine, the task to be scheduled must wait until the previous task to be scheduled is completely executed.

[0029] In addition, the combination of the virtual machine group and the host where each virtual machine in the virtual machine group is located (that is, the host group) can be in the same data center, so as to reduce the communication time between the host control component and the virtual machine. Correspondingly, each virtual machine in the virtual machine group in the embodiment can be created and accurately placed on the corresponding host in the same data center, so as to ignore the influence of the migration strategy of the virtual machine on the execution of the job task, and adopt a good topology structure, so that the communication time between the virtual machines and between the host control component and the virtual machines is almost 0. In the later scheduling optimization, the internal communication time and the energy consumption generated thereby can be ignored.

[0030] In the embodiment, the number of tasks to be scheduled in the task set to be scheduled can be greater than the number of virtual machines in the virtual machine group. That is, all job tasks (that is, tasks to be scheduled) submitted by the user at the same time can be collected into a global job list, and the number of submitted job tasks is much greater than the number of virtual machines. For the scheduler, the length of the global job list and the number of job tasks are known, and the scheduler can distribute the job tasks in the list to the corresponding virtual machines.

[0031] Step 102: determining the respective tasks to be scheduled corresponding to each virtual machine in the virtual machine group by using a double Q learning algorithm. Each task to be scheduled in the task set to be scheduled corresponds to a virtual machine, and the difference between the number of tasks to be scheduled corresponding to any two virtual machines is less than or equal to the balance threshold.

[0032] It can be understood that a double Q-learning algorithm can be used in this step to schedule the virtual machine resources, which can use two independent Q value functions to reduce the overestimation of the traditional Q-learning algorithm, select the optimal action to realize the scheduling of the business application to the virtual machine, and thus find a scheduling scheme that makes the total task completion time shorter and implement it. That is, in this embodiment, the double Q-learning algorithm is used to give a more stable and accurate scheduling strategy under the dynamic workload and resource competition scenario in the cloud computing environment, and the adaptability of the system in a complex dynamic environment is enhanced; and the average completion time of the business task can be reduced, and the quality of service demand of the user can be met.

[0033] Correspondingly, in this embodiment, three main aspects need to be considered when designing the model of the double Q-learning algorithm (i.e., the reinforcement learning model): state, action, and reward function; the state represents the perception of the agent (i.e., the model) to the environment, the action represents the decision made by the agent in the environment, and the reward function evaluates the decision result of the agent.

[0034] (1) State design: The state includes the state of the virtual machine and the state of the task to be scheduled. Assuming that there are n tasks to be scheduled and m virtual machines (n>m), the n tasks need to be reasonably allocated to different virtual machines of the host for processing. In this embodiment, the state space of the double Q-learning algorithm can be designed as a matrix ; wherein the value of d ij is 1 or 0, and d ij is 1, indicating that the i-th task to be scheduled (c i ) is allocated to the j-th virtual machine (v j ). In order to ensure the load balancing of the virtual machine, the difference in the number of tasks processed on all virtual machines cannot be greater than the balancing threshold (such as 1). n is the preset number of tasks, and m is the preset number of virtual machines.

[0035] (2) Action design: The action is the operation or decision made by the agent in a specific environment. In this embodiment, the agent needs to allocate different tasks to be scheduled to a certain number of virtual machines. Therefore, the allocation of each task to be scheduled is an action, and the size of the action space depends on the number of tasks to be scheduled. The agent selects an action to execute according to the current environment state.

[0036] Correspondingly, since the state design defines the matrix , wherein d ij indicates whether the task to be scheduled c i is allocated to the virtual machine v jThe agent selects a corresponding action according to the state of the matrix, that is, assigns the to-be-scheduled task to different virtual machines. The goal of the agent is to optimize the completion time of the task and the load balancing of the virtual machine by selecting a suitable action.

[0037] (3) Reward function design: the reward function is used to evaluate the goodness of the decision made by the agent, and therefore should be related to the optimization goal. In this embodiment, the optimization goal can be to optimize the completion time of all to-be-scheduled tasks, taking the completion time of the last to-be-scheduled in the cluster as the criterion. In a cloud computing environment, the execution time of each task can be different, and the speed of different virtual machines in processing tasks can also be different. Therefore, it is necessary to assign tasks of different lengths to appropriate virtual machines to reduce the total completion time of to-be-scheduled as much as possible and ensure the load balancing of virtual machines.

[0038] Correspondingly, in the training process of the agent, at each time step, the agent can select an action according to the current state and obtain a corresponding reward value according to the reward function; the selection of the action is based on the strategy of the agent (i.e., the double Q learning algorithm). At each time step, the agent will select the action with the maximum reward value as the next decision. Through continuous learning and optimization of action selection, the agent can gradually improve the effect of virtual machine resource scheduling.

[0039] For the total completion time of the to-be-scheduled task, the reward function of the double Q learning algorithm in this embodiment can be ; wherein C can represent a set of training scheduling tasks (i.e., a preset scheduling task training set) used in the training process; r can represent the reward feedback from the environment (i.e., the environment feedback reward), t c can represent the execution time of the training scheduling task c in the preset scheduling task training set. By summing the execution times of all tasks, the total time for processing all tasks is obtained. In reinforcement learning, the more optimal the action, the greater the reward, so a negative sign is added in the reward function; by using this reward function, the total completion time of the task can be effectively optimized.

[0040] It should be noted that for the double Q-learning algorithm used in this step, the specific manner of determining the respective virtual machines corresponding to each task to be scheduled in the virtual machine group can be set by the designer according to the practical scene and user demand, such as determining the current task to be scheduled from the current task set to be scheduled according to the preset state, the preset first Q value function table and the preset second Q value function table of the double Q-learning algorithm; the virtual machine corresponding to the target state-action pair is determined as the virtual machine corresponding to the current task to be scheduled. Among them, the current task to be scheduled is the target state-action pair corresponding to the target state-action pair in the current task set to be scheduled, and the target state-action pair is the state-action pair with the maximum average value of the first Q value and the second Q value corresponding to the target state-action pair in the current task set to be scheduled; the preset first Q value function table includes the first Q value corresponding to each state-action pair, the preset second Q value function table includes the second Q value corresponding to each state-action pair, and the state-action pair includes the corresponding relationship between each task to be scheduled in the task set to be scheduled and each virtual machine in the virtual machine group.

[0041] Among them, the above preset state, preset first Q value function table and preset second Q value function table can be the state space, first Q value function table (Q A value table) and second Q value function table (Q B value table) of the double Q-learning algorithm trained in advance. That is, in this step, the action with the maximum average value of the first Q value (Q A value) and the second Q value (Q B value) corresponding to the preset state can be selected from the current task set to be scheduled, and the virtual machine corresponding to the current task to be scheduled is determined, that is, the virtual machine corresponding to the state-action pair corresponding to the average value, so as to allocate the current task to be scheduled to the corresponding virtual machine.

[0042] Step 103: allocating all tasks to be scheduled in the task set to be scheduled to the respective virtual machines.

[0043] It can be understood that in this step, the respective virtual machines corresponding to each task to be scheduled determined according to the double Q-learning algorithm are used to allocate each task to be scheduled in the task set to be scheduled to the respective virtual machines, so as to realize the scheduling of virtual machine resources. As shown in Figure 2 , the task allocator can use the double Q-learning algorithm to allocate each business application (i.e. task to be scheduled) to the corresponding virtual machine.

[0044] Correspondingly, the embodiments are not limited to the logical sequence of step 102 and step 103. For example, step 102 can be performed first to determine the virtual machine corresponding to each of the to-be-scheduled tasks in the to-be-scheduled task set, and then step 103 can be performed to assign each of the to-be-scheduled tasks to the corresponding virtual machine. Alternatively, step 103 can be performed during the process of step 102, that is, after determining the virtual machine corresponding to the current to-be-scheduled task in step 102, the current to-be-scheduled task is assigned to the corresponding virtual machine by step 103 at the same time or before determining the virtual machine corresponding to the next to-be-scheduled task.

[0045] In the embodiments, the virtual machine resource scheduling method provided by the embodiments of the present application can determine the to-be-scheduled task corresponding to each virtual machine in the virtual machine group by using the double Q learning algorithm. The double Q learning algorithm has good representation capability, and can give more stable and accurate scheduling strategies under the dynamic work load and resource competition scenarios in the cloud computing environment, thereby enhancing the adaptability of the system in the complex dynamic environment. In addition, the average completion time of the business tasks can be reduced, and the quality of service requirements of the users can be met.

[0046] Based on the above embodiment, the virtual machine resource scheduling method provided by the embodiments can further include a training process of the double Q learning algorithm, such as the training process of the preset state, the preset first Q value function table, and the preset second Q value function table. Figure 3 As shown in the figure, the training process of the double Q learning algorithm can include: Step 201: initializing the first Q value function table, the second Q value function table, and the state space.

[0047] The state space is a matrix ; n is the number of training scheduling tasks in the preset scheduling task training set, and m is the number of preset virtual machines in the preset virtual machine group; d ij is 1 or 0, and d ij is 1 indicates that the i-th training scheduling task is assigned to the j-th virtual machine.

[0048] It can be understood that the state space S of the double Q learning algorithm in the embodiments is represented by the above-mentioned matrix , and each state represents a virtual machine resource scheduling scheme. The action space A can include n actions, that is, each state can have n actions; wherein n can be the number of training scheduling tasks in the preset scheduling task training set in the training process. In this step, two Q value function tables, i.e., the first Q value function table (Q A value table) and the second Q value function table (Q B value table) can be initialized; that is, the Q values (i.e., the first Q value and the second Q value) in the first Q value function table and the second Q value function table can be 0. Correspondingly, two Q value functions (Q A function and QB function) for the value of each state-action pair; set the learning variable γ, the step threshold, and the iteration threshold. The learning variable γ can be used to balance the weight of immediate rewards and future rewards, ranging from 0 to 1. When γ is close to 1, more emphasis is placed on future long-term rewards; when γ is close to 0, more emphasis is placed on current immediate rewards. The step threshold can be used to limit the maximum number of steps executed in each iteration, thereby controlling the length of a single training round and preventing infinite loops caused by an unterminated environment. The iteration threshold can be used to limit the maximum number of iterations in the training process.

[0049] Correspondingly, in this step, the state space can be initialized by randomly assigning states in the state space.

[0050] Step 202: Based on the current state space, use The greedy strategy selects the current training scheduling task and determines the preset virtual machine corresponding to the current training scheduling task.

[0051] The current training scheduling task is any training scheduling task in the preset scheduling task training set.

[0052] It can be understood that in this step, the current state of the state space (i.e., the current state) can be used to The greedy strategy selects the current training scheduling task (i.e. action a); if the probability Randomly select an action a (i.e., the current training scheduling task) to explore, with probability Select the corresponding Q A Value and Q B The action (i.e., state-action pair) with the largest average value is used to execute action a and allocate the current training scheduling task to the corresponding virtual machine.

[0053] Step 203: After allocating the current training scheduling task to the corresponding preset virtual machine, obtain the environmental feedback reward and the next state of the state space.

[0054] in, , r is the environmental feedback reward, C is the preset scheduling task training set, t c Schedule the execution time of task c for training.

[0055] Correspondingly, in this step, after assigning the training scheduling task to the corresponding preset virtual machine, the reward r of the environment feedback (i.e., the environment feedback reward) and the new state can be obtained. (ie the next state).

[0056] Step 204: Based on the environment feedback reward and the next state, the target Q value function table is updated according to the preset update selection probability to obtain an updated target Q value function table.

[0057] The target Q-value function table is the first Q-value function table or the second Q-value function table.

[0058] It can be understood that the preset update selection probability in this step can be a preset probability of selecting the target Q value function table from the first Q value function table or the second Q value function table, such as 0.5. In this step, the preset update selection probability can be used to select the target Q value function table. After the target Q value function table is selected, the environment feedback reward r and the next state are used. , update the target Q value function table.

[0059] For example, when the target Q value function table is the first Q value function table, the target action can be selected ; Calculate target value , update the first Q value function table (Q A Values ​​in the value table When the target Q value function table is the second Q value function table, you can select the target action ; Calculate target value , update the second Q value function table (Q B Values ​​in the value table . Among them, γ can be a pre-set learning variable, It can be a pre-set weight value; Can be expressed in the next state When executing action a (i.e., the current training scheduling task), Q A The value calculated by the function (i.e. Q A value); It can be said that in the current state s (that is, the state of the current state space), when action a is executed, Q A The value calculated by the function; It can be expressed that when updating the first Q value function table, by maximizing Select the next state The optimal action of Can be expressed in the next state Next, execute the target action When Q B The value calculated by the function (i.e. Q B value); correspondingly, 、 、 and The meaning can be set accordingly.

[0060] That is to say, the target Q value function table is the first Q value function table. In this step, the first Q value function table can be updated according to the environment feedback reward and the next state. ; The target Q value function table is the second Q value function table. In this step, the second Q value function table can be updated according to the environment feedback reward and next state. .

[0061] Step 205: Update the state space according to the next state.

[0062] Among them, in this step, the current state s of the state space can be updated to the next state .

[0063] Step 206: Determine whether this iteration has reached the single iteration termination condition; if not, proceed to step 202; if so, proceed to step 207.

[0064] The termination conditions for a single iteration include the completion of the allocation of the preset scheduling task training set and / or the number of training steps in this iteration reaching the step threshold.

[0065] It can be understood that in this step, the above steps 202 to 205 can be controlled to be repeated until the termination condition of a single iteration is reached, such as the number of training steps in this iteration reaches the step threshold (for example, the number of updates of the state space in this iteration reaches the step threshold) or the preset scheduling task training set is allocated (that is, all training scheduling tasks are allocated in this iteration).

[0066] Step 207: Determine whether the training completion condition is met; if not, proceed to step 202; if so, proceed to step 208.

[0067] The training completion conditions include that the number of iterations reaches an iteration threshold and / or the total completion time of the training scheduling task corresponding to this iteration is within a preset time range.

[0068] Correspondingly, in this step, the above steps 202 to 206 can be controlled to be repeated until the training completion conditions are met, such as the algorithm converges and obtains the optimal solution (for example, the total completion time of the training scheduling task corresponding to this iteration is within the preset time range) or the number of iterations reaches the iteration number threshold; correspondingly, when it is determined in this step to enter step 202, the number of iterations can be increased by 1 to enter the next iteration and reallocate the training scheduling tasks in the preset scheduling task training set again.

[0069] Step 208: Determine the current task space as the preset state, determine the current first Q-value function table as the preset first Q-value function table, and determine the current second Q-value function table as the preset second Q-value function table.

[0070] It can be understood that the trained task space, the first Q value function table and the second Q value function table in this step can be respectively taken as a preset state, a preset first Q value function table and a preset second Q value function table for subsequent actual virtual machine resource scheduling. After training, the two Q value function tables of the first Q value function table and the second Q value function table have learned the internal law of matching job tasks and virtual machines, and can adapt to scenarios with different numbers of tasks and virtual machines; that is, since the state space is designed as a matrix , it can dynamically adapt to n computing tasks and m virtual machines; the action space also depends on the number of tasks to be scheduled, and has dynamic scalability.

[0071] Based on the above embodiment, the virtual machine resource scheduling method provided by the embodiment can further include a virtual machine adjustment (scaling down or scaling up) process in the virtual machine group, so as to timely adjust the virtual machines in the virtual machine group through prediction of the resources required by the business software, facilitate subsequent allocation of virtual machine resources, and ensure the foresight and accuracy of resource allocation through the combination of prediction and scheduling.

[0072] Specifically, please refer to Figure 4 , Figure 4 for a flowchart of a resource prediction adjustment process provided by an embodiment of the application. The process can include: Step 301: Obtain usage resource data of business software corresponding to each preset computing resource index; wherein the preset computing resource index includes at least one of central processor usage rate, memory occupancy, storage usage and graphics processor usage rate.

[0073] In the embodiment, the preset computing resource index can be a pre-set computing resource index for predicting the resource usage (i.e. usage resource prediction data) of the business software in the future (i.e. after a preset time step). The embodiment does not limit the specific number and type of the preset computing resource index, such as at least one of central processor (CPU) usage rate, memory occupancy, storage usage and graphics processor (GPU) usage rate. For example, the number of preset computing resource indexes can be greater than or equal to 2, so as to predict the future resource usage of the business software from multiple dimensions. For example, the preset computing resource index can include CPU usage rate (%), memory occupancy (GB), storage usage (TB) and GPU usage rate (%).

[0074] Correspondingly, the embodiment does not limit the specific way of obtaining the usage resource data of the business software corresponding to each preset computing resource index, for example, the usage resource data (such as CPU usage rate, memory occupancy, storage usage and GPU usage rate) in the running of the business software is collected at a preset time interval (such as real time) in this step. Figure 5the service application resource demand history data in the historical data table), such as CPU usage, memory occupancy, storage usage, and GPU usage.

[0075] Further, the collected usage resource data can also be preprocessed in this step to obtain preprocessed usage resource data, so as to use the preprocessed usage resource data for subsequent prediction process. The preprocessing can include at least one of missing value filling processing, abnormal value detection correction processing, and data standardization processing. For example, the missing value filling processing can use a time series interpolation method (such as linear interpolation or cubic spline interpolation) to fill in the missing data; the abnormal value detection correction processing can identify and correct abnormal values based on the interquartile range (IQR) method or the rolling median method of the box plot; and the data standardization processing can scale the data to the range of [0, 1] or the range with a mean of 0 and a standard deviation of 1 through Min-Max standardization (dispersion standardization) or Z-Score (z-score, or standard score) standardization.

[0076] Step 302: According to the usage resource data, using the summation autoregressive moving average model corresponding to each preset computing resource index, the usage resource prediction data after the preset time step corresponding to each preset computing resource index is obtained.

[0077] It can be understood that in this embodiment, the summation autoregressive integrated moving average (ARIMA) model can be used to predict the computing resource demand of each preset computing resource index of the business software, so as to correspondingly adjust the virtual machines in the virtual machine group, and solve the problems of low resource allocation efficiency and insufficient prediction accuracy in the related art.

[0078] Correspondingly, for the specific manner of obtaining the usage resource prediction data after the preset time step corresponding to each preset computing resource index according to the usage resource data using the summation autoregressive moving average model corresponding to each preset computing resource index in this step, the designer can set it according to the practical scene and user demand, such as the summation autoregressive moving average model corresponding to a certain preset computing resource index (i.e. the current preset computing resource index) can be ; wherein the current preset computing resource index is any preset computing resource index, is the autoregressive parameter obtained by training, i.e. the preset autoregressive parameter; is the moving average parameter obtained by training, i.e. the preset moving average parameter; p is the preset autoregressive order, q is the preset moving average order, and T is the preset time step. ​​the use resource prediction data of the current preset computing resource index at the preset time step after the current time t, the use resource data (i.e., actual resource observation value) of the current preset computing resource index at the time step before the preset time step after the current time t, i.e., X can represent the actual use resource data, may represent the predicted use resource prediction data; the prediction error of the current preset computing resource index at the q time steps before the preset time step after the current time t.

[0079] Correspondingly, the embodiment is not limited to the specific size of each time step and the preset time step (i.e., T time steps), such as 1 hour, 1 day, or 1 week for each time step, to support multi-time scale prediction such as hour-level, day-level, and week-level.

[0080] It should be noted that the method provided in the embodiment can further include a training process of the summation autoregressive moving average model corresponding to each preset computing resource index; such as obtaining model training data corresponding to each preset computing resource index; training the initial summation autoregressive moving average model corresponding to each preset computing resource index according to the model training data, to obtain model parameters of the trained initial summation autoregressive moving average model, so as to construct the summation autoregressive moving average model corresponding to each preset computing resource index by using the model parameters.

[0081] Correspondingly, the process of obtaining the model training data corresponding to each preset computing resource index can include: obtaining historical resource use data of the business software corresponding to each preset computing resource index, and taking the historical resource use data as the model training data. It can also include: obtaining historical resource use data (such as business application resource demand historical data in Figure 6 ) of the business software corresponding to each preset computing resource index; pre-processing the historical resource use data to obtain the model training data; wherein the pre-processing includes at least one of missing value filling processing, abnormal value detection correction processing, and data standardization processing.

[0082] Correspondingly, for the above-mentioned method of training the initial sum autoregressive sliding average model corresponding to each preset computing power resource indicator based on the model training data and obtaining the model parameters of the trained initial sum autoregressive sliding average model, the designer can set it by himself according to the practical scenario and user needs. For example, the model training data can be divided according to a preset division ratio (such as 8:2) to obtain a training set and a test set corresponding to each preset computing power resource indicator; the training set is used to iteratively train the initial sum autoregressive sliding average model corresponding to each preset computing power resource indicator, and the target model parameters of the initial sum autoregressive sliding average model are adjusted by the maximum likelihood estimation algorithm to obtain the initial sum autoregressive sliding average model after iterative training; the test set is used to verify the initial sum autoregressive sliding average model corresponding to each preset computing power resource indicator after iterative training (such as Figure 6 ), obtain the model parameters of the initial summed autoregressive moving average model that has passed the verification.

[0083] For example, an initial ARIMA model can be constructed for each preset computing resource indicator; the initial ARIMA model of a preset computing resource indicator (i.e., the current preset computing resource indicator) can be ; Wherein, B is the preset difference operator, and d is the preset difference order; The model training data at time t (the current time) corresponding to the current preset computing resource indicator; is the preset error, that is, the prediction error corresponding to the model training data at time t; to is the autoregressive parameter to be trained; to is the moving average parameter to be trained; to That is, in this embodiment, the model parameters of the verified initial ARIMA model can be obtained by training and verifying the initial ARIMA model corresponding to the current preset computing power resource indicator, as shown above. to 、 to and to ;Thus, these model parameters are used to construct the ARIMA model corresponding to the current preset computing resource indicators, such as using to 、 to and to , respectively set the above to 、 to and and .

[0084] Correspondingly, for the specific verification process of verifying the initial summation autoregressive moving average model after iteration training corresponding to each preset computing resource index respectively, the designer can set it by himself, for example, according to the test set corresponding to the current preset computing resource index, the evaluation value of the initial summation autoregressive moving average model after iteration training corresponding to the current preset computing resource index can be determined by using , where MSE is the evaluation value, that is, using MSE (mean square error) as the evaluation value; n is the number of samples in the test set corresponding to the current preset computing resource index, is the usage resource prediction data of the i th sample, is the future real data of the usage resource of the i th sample; when the evaluation value reaches the evaluation threshold, it is determined that the initial summation autoregressive moving average model corresponding to the current preset computing resource index passes the verification. Correspondingly, if the evaluation value does not reach the evaluation threshold, the parameters (such as model parameters) can be adjusted or the data features can be increased.

[0085] Further, the method provided in the embodiment can further include determining the preset difference order, the preset autoregressive order and the preset moving average order in the preset search range by grid search, using Akaike information criterion (AIC) or Bayesian information criteria (BIC); wherein the preset search range is {0, 1, 2, 3}. For example, by grid search combined with AIC or BIC for parameter optimization, the search range is p, d, q ∈ {0, 1, 2, 3}, and the (p, d, q) combination that makes the value of AIC or BIC minimum is selected.

[0086] Step 303: adjusting the virtual machines in the virtual machine group according to the usage resource prediction data.

[0087] It can be understood that in the embodiment, the virtual machines in the virtual machine group processing the job tasks (such as the above-mentioned to-be-scheduled tasks) of the business software can be adjusted, such as expansion or contraction, according to the usage resource prediction data after the preset time step corresponding to each preset computing resource index.

[0088] For example, in this step, the computing resource demand curve corresponding to each preset computing resource index can be generated; thereby the virtual machines in the virtual machine group are adjusted according to the computing resource demand curve, realizing dynamic resource allocation. Correspondingly, in the embodiment, the usage resource prediction data can also be used for GPU cluster scheduling and adjustment of other related computing resources.

[0089] Further, the virtual machine group can not only include deployed virtual machines (i.e., deployed virtual machines), but also include undeployed virtual machines, i.e., virtual machines determined to need to be expanded by using resource prediction data, so as to allocate virtual machine resources in the virtual machine group before or during expansion of the virtual machine group. Correspondingly, the virtual machine group can not include virtual machines to be deleted, i.e., virtual machines determined to need to be deleted by using resource prediction data.

[0090] In this embodiment, by using the ARIMA model, the virtual machine group can be expanded or contracted based on the predicted resource demand at the future time point, so as to realize the ARIMA-DQL-based heterogeneous computing power scheduling strategy, make the computing power resource allocation more reasonable, comprehensively improve the resource utilization, guarantee the business response speed, enhance the system scalability, and provide an innovative solution for intelligent and efficient management of resources in a cloud computing environment, which can effectively balance performance, cost and stability.

[0091] Corresponding to the above method embodiment, the present embodiment also provides an updating device for a cluster global routing table. The updating device for a cluster global routing table described below can be referred to in conjunction with the updating method for a cluster global routing table described above.

[0092] Please refer to Figure 7 , Figure 7 A structural block diagram of a virtual machine resource scheduling device provided by the present embodiment is shown in FIG. 1. The device can include: A task acquisition module 10 is configured to acquire a set of to-be-scheduled tasks of a business software. The number of to-be-scheduled tasks in the set of to-be-scheduled tasks is greater than the number of virtual machines in the virtual machine group, and the resource amount of each virtual machine satisfies the demand resource amount of any to-be-scheduled task. A task determination module 20 is configured to determine respective to-be-scheduled tasks corresponding to each virtual machine in the virtual machine group by using a double Q learning algorithm. Each to-be-scheduled task in the set of to-be-scheduled tasks corresponds to one virtual machine, and the difference between the number of to-be-scheduled tasks corresponding to any two virtual machines is less than or equal to an equalization threshold. A task allocation module 30 is configured to allocate all to-be-scheduled tasks in the set of to-be-scheduled tasks to respective virtual machines.

[0093] In some embodiments, the task determination module 20 can include: The task determining submodule is configured to determine a current to-be-scheduled task from the current to-be-scheduled task set according to a preset state of the double Q learning algorithm, a preset first Q value function table and a preset second Q value function table; the current to-be-scheduled task is a to-be-scheduled task corresponding to a target state-action pair with the largest average value of a first Q value and a second Q value in respective state-action pairs of each to-be-scheduled task in the current to-be-scheduled task set; the preset first Q value function table includes respective first Q values of the state-action pairs, the preset second Q value function table includes respective second Q values of the state-action pairs, and the state-action pair includes a corresponding relationship between each to-be-scheduled task in the to-be-scheduled task set and each virtual machine in the virtual machine group; The virtual machine determining submodule is configured to determine, by the determining submodule, a virtual machine corresponding to the target state-action pair as a virtual machine corresponding to the current to-be-scheduled task.

[0094] In some embodiments, the apparatus further includes: The Q value initializing module is configured to initialize the first Q value function table, the second Q value function table and the state space; the state space is a matrix ; n is a number of training scheduling tasks in the preset scheduling task training set, m is a number of preset virtual machines in the preset virtual machine group, and d ij is a value of 1 or 0, and d ij 1 indicates that the i-th training scheduling task is assigned to the j-th virtual machine. The selection determining module is configured to select, according to the current state space, the current training scheduling task by using a greedy strategy and determine a preset virtual machine corresponding to the current training scheduling task; the current training scheduling task is any training scheduling task in the preset scheduling task training set. The assignment obtaining module is configured to obtain an environment feedback reward and a next state of the state space after assigning the current training scheduling task to the corresponding preset virtual machine; wherein , r is the environment feedback reward, C is the preset scheduling task training set, t c is an execution time of the training scheduling task c. The Q value updating module is configured to update the target Q value function table according to the environment feedback reward and the next state and according to a preset update selection probability, to obtain an updated target Q value function table; the target Q value function table is the first Q value function table or the second Q value function table. The state updating module is configured to update the state space according to the next state. The iteration judgment module is configured to judge whether the current iteration reaches a single-iteration termination condition; if the single-iteration termination condition is not reached, the selection determination module is sent a start signal; wherein the single-iteration termination condition includes a preset dispatch task training set allocation completion and / or a training step number of the current iteration reaching a step number threshold; The training judgment module is configured to judge whether a training completion condition is met if the single-iteration termination condition is reached; if the training completion condition is not met, the selection determination module is sent a start signal; wherein the training completion condition includes an iteration number reaching an iteration number threshold and / or a total completion time of a training dispatch task corresponding to the current iteration being within a preset time range; The completion determination module is configured to determine the current task space as a preset state, the current first Q value function table as a preset first Q value function table, and the current second Q value function table as a preset second Q value function table if the training completion condition is met.

[0095] In some embodiments, the apparatus further includes: The data acquisition module is configured to acquire usage resource data of the business software corresponding to each preset computing resource index; wherein the preset computing resource index includes at least one of a central processing unit usage rate, a memory occupancy, a storage usage, and a graphics processing unit usage rate; The resource prediction module is configured to obtain, according to the usage resource data, usage resource prediction data of each preset computing resource index after a preset time step by using a summation autoregressive moving average model corresponding to each preset computing resource index. The virtual machine adjustment module is configured to adjust the virtual machines in the virtual machine group according to the usage resource prediction data.

[0096] In some embodiments, the summation autoregressive moving average model corresponding to the current preset computing resource index is ; wherein the current preset computing resource index is any preset computing resource index, is a trained autoregressive parameter, is a trained moving average parameter, p is a preset autoregressive order, q is a preset moving average order, and T is a preset time step, is the usage resource prediction data of the current preset computing resource index after the preset time step, is the usage resource data of the current preset computing resource index at a time step before the preset time step after the current time t, is the prediction error of the current preset computing resource index at the q time steps before the preset time step after the current time t.

[0097] In some embodiments, the apparatus can further include:​​ The training acquisition module is configured to acquire model training data corresponding to each of the preset computing resource indicators; The model training module is configured to train the initial summation autoregressive moving average model corresponding to each of the preset computing resource indicators according to the model training data, to obtain model parameters of the trained initial summation autoregressive moving average model, and to construct a summation autoregressive moving average model corresponding to each of the preset computing resource indicators by using the model parameters.

[0098] In some embodiments, the training acquisition module can include: The history acquisition submodule is configured to acquire historical resource usage data of the business software corresponding to each of the preset computing resource indicators; The preprocessing submodule is configured to preprocess the historical resource usage data to obtain the model training data; and the preprocessing includes at least one of missing value filling processing, abnormal value detection and correction processing, and data standardization processing.

[0099] In some embodiments, the model training module can include: The division submodule is configured to divide the model training data according to a preset division ratio to obtain a training set and a test set corresponding to each of the preset computing resource indicators; The training submodule is configured to iteratively train the initial summation autoregressive moving average model corresponding to each of the preset computing resource indicators by using the training set, and to adjust the target model parameters of the initial summation autoregressive moving average model by using a maximum likelihood estimation algorithm to obtain an iteratively trained initial summation autoregressive moving average model; The verification submodule is configured to verify the iteratively trained initial summation autoregressive moving average model corresponding to each of the preset computing resource indicators by using the test set to obtain model parameters of an initial summation autoregressive moving average model that passes the verification.

[0100] In some embodiments, the initial summation autoregressive moving average model corresponding to the current preset computing resource indicator is ; is a preset error, B is a preset difference operator, and d is a preset difference order.

[0101] In some embodiments, the device further includes: The parameter search module is configured to determine the preset difference order, the preset autoregressive order, and the preset moving average order in a preset search range by using a grid search, Akaike information criterion, or Bayesian information criterion; and the preset search range is {0, 1, 2, 3}.

[0102] In some embodiments, the verification submodule can include: The evaluation unit is configured to determine an evaluation value of the initial summation autoregressive moving average model after iteration training corresponding to the current preset computing resource index according to a test set corresponding to the current preset computing resource index , wherein the MSE is the evaluation value, n is a sample number of the test set corresponding to the current preset computing resource index, is the usage resource prediction data of the ith sample, is the future real data of the usage resource of the ith sample; The verification unit is configured to determine that the initial summation autoregressive moving average model corresponding to the current preset computing resource index passes the verification if the evaluation value reaches an evaluation threshold.

[0103] In some embodiments, the virtual machines in the virtual machine group include deployed virtual machines and undeployed virtual machines, and the undeployed virtual machines are virtual machines that need to be expanded according to the usage resource prediction data.

[0104] In this embodiment, the task determination module 20 determines the respective tasks to be scheduled for each virtual machine in the virtual machine group by using the double Q learning algorithm. The double Q learning algorithm has good representation capability, and can give more stable and accurate scheduling strategies under the dynamic workload and resource competition scenarios in the cloud computing environment, thereby enhancing the adaptability of the system in a complex dynamic environment. In addition, the average completion time of business tasks can be reduced, and the quality of service requirements of users can be met.

[0105] Corresponding to the above method embodiments, the embodiments of the present application also provide a virtual machine resource scheduling device. The virtual machine resource scheduling device described below can be mutually corresponding with reference to the virtual machine resource scheduling method described above.

[0106] Please refer to Figure 8 , Figure 8 FIG. 1 is a structural schematic diagram of a virtual machine resource scheduling device provided by the embodiments of the present application. The virtual machine resource scheduling device can include: a memory D1 configured to store a computer program; a processor D2 configured to execute the computer program to implement the steps of the virtual machine resource scheduling method provided by the method embodiments.

[0107] In this embodiment, the virtual machine resource scheduling device is specifically a server or a computer device, such as a server of a cloud computing system.

[0108] Corresponding to the above method embodiments, the embodiments of the present application also provide a computer program product. The computer program product described below can be mutually corresponding with reference to the virtual machine resource scheduling method described above.

[0109] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the virtual machine resource scheduling method provided in the above method embodiment.

[0110] Corresponding to the above method embodiment, an embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium described below and the virtual machine resource scheduling method described above can refer to each other.

[0111] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the virtual machine resource scheduling method of the above method embodiment.

[0112] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, which may store program codes.

[0113] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. References to the common and similar parts between the various embodiments are sufficient. The devices, apparatuses, computer program products, and computer-readable storage media disclosed in the embodiments are described briefly because they correspond to the methods disclosed in the embodiments. For relevant details, refer to the description of the methods.

[0114] The above is a detailed introduction to the virtual machine resource scheduling method, device, equipment and computer-readable storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A virtual machine resource scheduling method, characterized in that: include: Obtaining a set of tasks to be scheduled for the business software; wherein the number of tasks to be scheduled in the set of tasks to be scheduled is greater than the number of virtual machines in the virtual machine group, and the resource amount of each virtual machine meets the required resource amount of any task to be scheduled; Determine, using a double-Q learning algorithm, tasks to be scheduled corresponding to each virtual machine in the virtual machine group; wherein, each task to be scheduled in the task set corresponds to a virtual machine, and the difference in the number of tasks to be scheduled corresponding to any two virtual machines is less than or equal to a balance number threshold; All tasks to be scheduled in the task set to be scheduled are respectively allocated to their corresponding virtual machines.

2. The virtual machine resource scheduling method according to claim 1, wherein: The determining of the to-be-scheduled tasks corresponding to each virtual machine in the virtual machine group by using the double-Q learning algorithm includes: According to the preset state, the preset first Q value function table and the preset second Q value function table of the dual Q learning algorithm, the current task to be scheduled is determined from the current task set to be scheduled; wherein the current task to be scheduled is the task to be scheduled corresponding to the target state-action pair in the current task set to be scheduled, and the target state-action pair is a state-action pair with the largest average value of the first Q value and the second Q value corresponding to each state-action pair corresponding to each task to be scheduled in the current task set to be scheduled; the preset first Q value function table includes the first Q value corresponding to each state-action pair, the preset second Q value function table includes the second Q value corresponding to each state-action pair, and the state-action pair includes the corresponding relationship between each task to be scheduled in the task set to be scheduled and each virtual machine in the virtual machine group; The virtual machine corresponding to the target state-action pair is determined as the virtual machine corresponding to the current task to be scheduled.

3. The virtual machine resource scheduling method according to claim 2, wherein: Also includes: Initialize the first Q value function table, the second Q value function table and the state space; wherein the state space is a matrix ; n is the number of training scheduling tasks in the preset scheduling task training set, m is the number of preset virtual machines in the preset virtual machine group; d ij The value is 1 or 0, d ij A value of 1 indicates that the i-th training scheduling task is assigned to the j-th virtual machine; According to the current state space, use The greedy strategy selects the current training scheduling task and determines the preset virtual machine corresponding to the current training scheduling task; wherein the current training scheduling task is any of the training scheduling tasks in the preset scheduling task training set; After assigning the current training scheduling task to the corresponding preset virtual machine, obtain the environmental feedback reward and the next state of the state space; , r is the environmental feedback reward, C is the preset scheduling task training set, t c Schedule the execution time of task c for training; According to the environmental feedback reward and the next state, the target Q-value function table is updated according to the preset update selection probability to obtain an updated target Q-value function table; wherein the target Q-value function table is the first Q-value function table or the second Q-value function table; Update the state space according to the next state; Determine whether the current iteration meets the single iteration termination condition; wherein the single iteration termination condition includes that the preset scheduling task training set is allocated and / or the number of training steps in this iteration reaches the step threshold; If the single iteration termination condition is met, it is determined whether the training completion condition is met; wherein the training completion condition includes that the number of iterations reaches the iteration number threshold and / or the total completion time of the training scheduling task corresponding to this iteration is within a preset time range; If the single iteration termination condition is not reached or the training completion condition is not met, the current state space is used to execute the The greedy strategy selects the current training scheduling task and determines the steps of the preset virtual machine corresponding to the current training scheduling task; If the training completion condition is met, the current task space is determined as the preset state, the current first Q value function table is determined as the preset first Q value function table, and the current second Q value function table is determined as the preset second Q value function table.

4. The virtual machine resource scheduling method according to claim 1, wherein: Also includes: Obtaining resource usage data of the business software corresponding to each preset computing resource indicator; wherein the preset computing resource indicator includes at least one of a central processing unit utilization rate, a memory usage rate, a storage usage rate, and a graphics processing unit utilization rate; According to the resource usage data, using the summed autoregressive sliding average model corresponding to each of the preset computing power resource indicators, obtaining resource usage prediction data after a preset time step corresponding to each of the preset computing power resource indicators; The virtual machines in the virtual machine group are adjusted according to the resource usage prediction data.

5. The virtual machine resource scheduling method according to claim 4, wherein: The summed autoregressive moving average model corresponding to the current preset computing power resource indicator is: ; Wherein, the current preset computing power resource indicator is any of the preset computing power resource indicators, to is the autoregressive parameter obtained through training, to is the moving average parameter obtained through training, p is the preset autoregressive order, q is the preset sliding average order, T is the preset time step, The resource usage forecast data for the current preset computing power resource indicator at the next preset time step, The resource usage data for the current preset computing power resource indicator at the time step before the preset time step after the current time t. It is the prediction error of the current preset computing power resource indicator at the first q time steps of the preset time step after the current time t.

6. The virtual machine resource scheduling method according to claim 5, characterized in that: Also includes: Obtain model training data corresponding to each of the preset computing power resource indicators; According to the model training data, the initial sum autoregressive sliding average model corresponding to each of the preset computing power resource indicators is trained to obtain the model parameters of the trained initial sum autoregressive sliding average model, so as to use the model parameters to construct the sum autoregressive sliding average model corresponding to each of the preset computing power resource indicators.

7. The virtual machine resource scheduling method according to claim 6, characterized in that: The obtaining of model training data corresponding to each of the preset computing resource indicators includes: Obtaining historical resource usage data of the business software corresponding to each of the preset computing power resource indicators; The historical resource usage data is preprocessed to obtain the model training data; wherein the preprocessing includes at least one of missing value filling processing, outlier detection and correction processing, and data standardization processing.

8. The virtual machine resource scheduling method according to claim 6, characterized in that: The initial sum autoregressive moving average model corresponding to each of the preset computing power resource indicators is trained according to the model training data to obtain model parameters of the trained initial sum autoregressive moving average model, including: Divide the model training data according to a preset division ratio to obtain a training set and a test set corresponding to each preset computing resource indicator; Iteratively training the initial sum autoregressive moving average model corresponding to each of the preset computing power resource indicators using the training set, and adjusting the target model parameters of the initial sum autoregressive moving average model using a maximum likelihood estimation algorithm to obtain the initial sum autoregressive moving average model after iterative training; The test set is used to verify the initial sum autoregressive sliding average model corresponding to each of the preset computing power resource indicators after iterative training, and the model parameters of the verified initial sum autoregressive sliding average model are obtained.

9. The virtual machine resource scheduling method according to claim 8, characterized in that: The initial sum autoregressive moving average model corresponding to the current preset computing power resource indicator is: ; is the preset error, B is the preset difference operator, and d is the preset difference order.

10. The virtual machine resource scheduling method according to claim 9, characterized in that: Also includes: The preset difference order, the preset autoregressive order and the preset sliding average order are determined within a preset search range by using a grid search and the Akaike information criterion or the Bayesian information criterion; wherein the preset search range is {0, 1, 2, 3}.

11. The virtual machine resource scheduling method according to claim 8, characterized in that: The method of using the test set to respectively verify the iteratively trained initial sum autoregressive moving average model corresponding to each of the preset computing power resource indicators to obtain the model parameters of the verified initial sum autoregressive moving average model includes: According to the test set corresponding to the current preset computing power resource indicators, use , determine the evaluation value of the initial sum autoregressive sliding average model after iterative training corresponding to the current preset computing power resource indicator; where MSE is the evaluation value, n is the number of samples in the test set corresponding to the current preset computing power resource indicator, is the resource usage prediction data of the i-th sample, The future real data of the resource usage of the i-th sample; If the evaluation value reaches the evaluation threshold, it is determined that the initial summed autoregressive sliding average model corresponding to the current preset computing power resource indicator has passed the verification.

12. The virtual machine resource scheduling method according to claim 4, wherein: The virtual machines in the virtual machine group include deployed virtual machines and undeployed virtual machines, and the undeployed virtual machines are virtual machines that need to be expanded and increased as determined by using the resource usage prediction data.

13. A virtual machine resource scheduling device, characterized in that: include: A task acquisition module is used to obtain a set of tasks to be scheduled for the business software; wherein the number of tasks to be scheduled in the set of tasks to be scheduled is greater than the number of virtual machines in the virtual machine group, and the resource amount of each virtual machine meets the required resource amount of any task to be scheduled; a task determination module, configured to determine, using a double-Q learning algorithm, tasks to be scheduled corresponding to each virtual machine in the virtual machine group; wherein each task to be scheduled in the task set corresponds to a virtual machine, and the difference in the number of tasks to be scheduled corresponding to any two virtual machines is less than or equal to a balance number threshold; The task allocation module is used to allocate all the tasks to be scheduled in the task set to be scheduled to their corresponding virtual machines respectively.

14. A virtual machine resource scheduling device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the virtual machine resource scheduling method according to any one of claims 1 to 12 when executing the computer program.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the virtual machine resource scheduling method according to any one of claims 1 to 12 are implemented.

Images