An Optimal Scheduling Method for Multi-Server Jobs

Abstract

The present invention discloses an optimization scheduling method for multi-server jobs, which relates to the technical field related to cloud computing, including proposing a general bipartite graph-based group scheduling model, with the optimization goal of maximizing Asw; based on the model, a framework for learning the expectation and variance of MSJ running time on different servers through historical reward data is proposed; based on the learning framework, an Asw maximization strategy based on a genetic algorithm is proposed, which can maximize the overall social benefit while ensuring that various types of constraints are established. For multi-server jobs, the present invention proposes a group scheduling method that can actively learn the completion rate of jobs and maximize the cumulative social benefits at the same time, which can optimize the cumulative social benefits, balance the interests of all parties in society, and pursue the maximization of the overall social benefits. The designed scheduling strategy can significantly save computing costs for enterprises and improve the production-to-revenue ratio.

Description

Technical Field

[0001] The present invention relates to the technical field related to cloud computing, and in particular to an optimization scheduling method for multi-server operations. Background Art

[0002] Today's computing clusters are full of various "Multi-Server Jobs (MSJ)". MSJ has multiple tasks that need to run simultaneously. The running of these tasks requires the use of multiple servers' CPU cores, GPUs, FPGA boards or other coprocessors. Taking the cluster-data dataset released by Google as an example, 90% of the tasks require more than one CPU core, and nearly 20% of the tasks require no less than 1,000 CPU cores. For an MSJ, all its tasks must be correctly scheduled and started at the same time for it to run normally.

[0003] In the Chinese invention patent application number: CN201910424159.9, there is disclosed a decision on unit combination scheduling taking into account user utility. Although the decision on unit combination scheduling taking into account user utility comprehensively considers the benefits and costs of users participating in demand response, and incorporates demand response compensation into the decision of analyzing social benefits, that is, the compensation mechanism for users participating in DR is incorporated into the unit combination optimization decision considering user utility, and a unit combination optimization strategy taking into account user utility compensation is established to maximize social benefits. However, for multi-server operations in a distributed cloud computing cluster, how to maximize service quality or group benefits under the conditions that the task execution time cannot be accurately predicted and the group scheduling principle needs to be followed.

[0004] Therefore, it is necessary to propose an optimal scheduling method for multi-server jobs to solve the above problems. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides an optimized scheduling method for multi-server jobs, which solves the problem of how to maximize the service quality or group benefit for multi-server jobs in a distributed cloud computing cluster under the conditions that the task execution time cannot be accurately predicted and the group scheduling principles need to be complied with.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A multi-server operation optimization scheduling method comprises the following steps:

[0010] 1) A general bipartite graph-based group scheduling model is proposed, with the optimization objective of maximizing Asw. MSJ includes the composition of parallel deep learning training tasks and the workflow of parallel deep learning training;

[0011] 2) Based on this model, a framework is proposed to learn the expectation and variance of MSJ running time on different servers through historical reward data. The scheduling of MSJ is called group scheduling. According to the different input data, the scheduling problem can be divided into the following two categories: batch job scheduling and streaming job scheduling;

[0012] 3) Based on this learning framework, a genetic algorithm-based Asw maximization strategy is proposed, which can maximize the overall social benefits while ensuring that various types of constraints are established.

[0013] Optionally, the parallel deep learning training task group consists of: a parameter server (ParameterServer, a process running on a physical machine or a virtual machine, hereinafter referred to as PS), and multiple worker nodes (Worker, a process that performs back propagation).

[0014] Optionally, the workflow of the parallel deep learning training is: in each round of iteration, each Worker updates its local gradient through the back propagation algorithm according to its own local data set, and then transmits the local gradient to the PS through the network.

[0015] Optionally, after the PS collects all local gradients, it obtains the global gradient of this round by taking the average value, and the PS transmits the global gradient to each Worker through the network to enter the next round of iteration.

[0016] Optionally, the batch job refers to a task that "requires only a single input and automatically releases resources after the operation is completed". After these tasks arrive at different times, the scheduler needs to specify a server that can run each task and allocate computing resources to the task (such as a CPU core, 500Mi of memory). When specifying a server, the scheduler can only select a server that can meet the resource requirements. Taking into account the active release of occupied resources after the job is completed, the scheduling strategy usually aims to minimize the completion time of all jobs (the time it takes from the arrival of the first job to the completion of the last job).

[0017] Optionally, the streaming job refers to a job in which different data will be input in different time windows to perform corresponding calculations. A typical streaming job is real-time big data analysis. These jobs must always be running and need to process real-time input data. At this time, what needs to be scheduled is the input data stream. The scheduling strategy needs to decide how to distribute the input data stream to one of the multiple background tasks that can process the data stream, which is called load balancing.

[0018] (III) Beneficial effects

[0019] The present invention provides a multi-server operation optimization scheduling method, which has the following beneficial effects:

[0020] 1. The present invention proposes a group scheduling method for multi-server operations that can actively learn the completion rate of the operation and maximize the cumulative social benefits at the same time.

[0021] 2. The present invention can optimize the accumulated social benefits, balance the interests of all parties in society, and pursue the maximization of the overall social benefits. The designed scheduling strategy can significantly save computing costs for enterprises and improve production-return ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a multi-server job scheduling scenario based on a bipartite graph of the structure of the present invention;

[0023] Figure 2 A schematic diagram of the flow of the scheduling algorithm of the structure of the present invention;

[0024] Figure 3 Schematic diagram of the genetic algorithm subroutine flow chart of the structure of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] According to Figure 1-3 As shown, the present invention provides a technical solution:

[0027] A multi-server operation optimization scheduling method comprises the following steps:

[0028] 1) A general bipartite graph-based group scheduling model is proposed, with the optimization objective of maximizing Asw. MSJ includes the composition of parallel deep learning training tasks and the workflow of parallel deep learning training;

[0029] 2) Based on this model, a framework is proposed to learn the expectation and variance of MSJ running time on different servers through historical reward data. The scheduling of MSJ is called group scheduling. According to the different input data, the scheduling problem can be divided into the following two categories: batch job scheduling and streaming job scheduling;

[0030] 3) Based on this learning framework, a genetic algorithm-based Asw maximization strategy is proposed, which can maximize the overall social benefits while ensuring that various types of constraints are established.

[0031] As an optional technical solution of the present invention: the parallel deep learning training task group is composed of: a parameter server (Parameter Server, which is a process running on a physical machine or a virtual machine, hereinafter referred to as PS), and multiple worker nodes (Worker, which is a process that performs back propagation).

[0032] As an optional technical solution of the present invention: the workflow of the parallel deep learning training is: in each round of iteration, each Worker updates its local gradient through the back propagation algorithm according to its own local data set, and then transmits the local gradient to the PS through the network.

[0033] As an optional technical solution of the present invention: after the PS collects all local gradients, it obtains the global gradient of this round by taking the average value. The PS transmits the global gradient to each Worker through the network to enter the next round of iteration. It can be seen that this job can only run normally when the PS and at least one Worker are successfully scheduled and started.

[0034] As an optional technical solution of the present invention: the batch processing job refers to a task that "requires only a single input and automatically releases resources after the operation is completed". After these tasks arrive at different times, the scheduler needs to specify a server that can run each task and allocate computing resources to the task (such as a CPU core, 500Mi of memory). When specifying a server, the scheduler can only select a server that can meet the resource requirements. Taking into account the active release of occupied resources after the job is completed, the scheduling strategy usually aims to minimize the completion time of all jobs (the time it takes from the arrival of the first job to the completion of the last job).

[0035] As an optional technical solution of the present invention: the streaming job refers to a job in which different data will be input in different time windows to perform corresponding calculations. Typical streaming jobs are real-time big data analysis. These jobs must always be in running state and need to process real-time input data. At this time, what needs to be scheduled is the input data stream. The scheduling strategy needs to decide whether to distribute the input data stream to one of the multiple background tasks that can process the data stream, which is called load balancing.

[0036] In step (1), a general heterogeneous server cluster R is considered. There are multiple servers in the cluster. Let r∈R represent a server. Consider K different types of computing devices, such as CPU cores, GPUs, NPUs, FPGAs, etc. Each server is installed with different types and numbers of computing devices. For server r, the number of the kth computing device installed on it is c (k，r) , using the vector c = [c (k，r) ] k∈K，r∈R Indicates the number of various computing devices installed on each server in the cluster.

[0037] The present invention considers general multi-server jobs, each of which is composed of at least one task. The type set of multi-server jobs is denoted as L, and l∈L represents a specific type of multi-server job. The differences between different types of multi-server jobs are mainly reflected in the following two aspects: First, they may have different requirements for different types of resources. For example, parallel deep learning training jobs may have greater requirements for GPUs, while graph computing tasks may have greater requirements for memory and CPUs; Second, they may have different numbers and types of task components. The number of tasks of job type l is denoted as Q l , for any task q∈Q l , let the number of its demands for the kth resource be

[0038] The scheduling model is modeled as a bipartite graph (L, R, E). In the bipartite graph (L, R, E), L is the set of types of multi-server jobs, R is the set of heterogeneous servers in the cluster, and E is the edge in the bipartite graph. Its physical meaning is: any edge e=(l, r)∈E exists if and only if the lth type of job can be distributed to the rth server for execution. The job type is called a "port" waiting for the job to arrive, and the edge is called a "channel", which is the path where a single task of each job can be distributed. An edge e=(l, r)∈E exists if and only if the following two conditions can be met at the same time: server r meets the node affinity of job type l; the number of computing devices equipped on node r meets the requirements of job type l.

[0039] Among them, node affinity means that due to business needs, job type l must be forwarded to certain specific server nodes for execution, and server r is among them. In the actual production environment, not all l∈L can be connected to all r∈R. The present invention calls this phenomenon "service constraint".

[0040] The present invention considers a continuous time domain T. Specifically, the time domain is divided into time slices t∈T of equal length. In each time slice, there is only one job of this type arriving at each port. Then, the scheduler needs to calculate the forwarding destination of each task of each job according to the scheduling strategy and send the task to it. For the length of the time slice, each job is required to be completed within a single time slice. For multi-server jobs, the scheduling behavior is for each task of the job. In each time slice t, record is the scheduling decision variable, where q∈Q l , which means whether the qth task of job type l is distributed to server r for execution, using vector Indicates that all scheduling variables need to be decided within time slice t.

[0041] Optimization objectives and constraints. In each time slice t, for each k∈K, r∈R, the scheduling decision x(t) needs to satisfy the following computing resource constraints:

[0042]

[0043] As mentioned above, the optimization goal of the scheduling strategy involved is to accumulate social benefits. Social Welfare comes from game economics theory and is the sum of the economic benefits of both parties to the transaction. In the bipartite graph scheduling problem involved, social benefit is the sum of the benefits of all job submitters and cluster maintainers. Specifically, in each time slice t, the task arriving at each port is submitted by the user of the cluster, and the cluster comes from the data center of the infrastructure provider. In this process, users need to pay corresponding financial costs to submit tasks and apply for cluster resources. If the task is successfully scheduled and executed, the user will obtain the corresponding psychological estimated benefit (valuation). The psychological estimated benefit here comes from game economics theory and is a quantification of the benefits a user obtains after purchasing an item. If the task is not successfully scheduled, the corresponding financial cost will also be refunded. The provider of the cluster will obtain payment from the user, but at the same time, it needs to pay the financial cost of cluster maintenance (electricity bills, management fees, etc.).

[0044] The user's psychological estimated benefit is modeled as a variable proportional to the average running speed of the job. As mentioned above, the running time of a single task of a job on any server is unknown and will fluctuate around a certain moving average. To solve this problem, it is assumed that the running speed of a single task on any server follows a potential and unknown distribution. Specifically, the distribution of the running rate of the qth task of job type l on server r is recorded as The expected value of this distribution is recorded as Obviously, as a data distribution, its expectation and variance can be learned from historical data, and the scheduling strategy involved in the present invention will also be developed based on this point.

[0045] At each time slice t, for the task submitter, the economic benefit generated at port l is modeled as

[0046]

[0047] U l (t) consists of two parts. The first part is the psychological expectation valuation obtained by the user when the task is successfully scheduled and run, and the second part is the financial cost paid by the user. is a random variable in time slice t, and its expected value is 1{p} is an indicator function. When the internal predicate p holds, 1{p} takes the value of 1, otherwise 1{p} takes the value of 0. l (t) is the financial cost that the task submitter needs to pay at port l. It means that each task q of job type l is distributed to at least one server to run. This means that every task q of job type l is successfully scheduled, satisfying the group scheduling rules.

[0048] At each time slice t, the economic benefits for the cluster providers are modeled as

[0049]

[0050] U c (t) also consists of two parts. The first part is the fee collected from each user submitter, and the second part is the cluster operation and maintenance cost, where f k (·) is the maintenance cost of the kth resource.

[0051] At each time slice t, the social benefit is modeled as the sum of the two economic benefits mentioned above:

[0052]

[0053] Therefore, the multi-server job scheduling problem can be modeled as:

[0054]

[0055] st (1)

[0056] in To perform the expectation operation.

[0057] The multi-server job scheduling problem described in step (1) is a stochastic optimization problem with random variables in the optimization objective. To solve this problem, step (2) proposes an expectation learning framework based on historical data, which continuously estimates the expected value through the specific benefits of historical time slices.

[0058] To simplify the description, define for

[0059]

[0060] In addition, Recorded as the number of times the qth task of job type l is distributed to server r for execution from the 1st time slice to the t-1th time slice:

[0061]

[0062] Expected Learning Framework, for expected value Design the following formula to estimate it:

[0063]

[0064] The meaning of the above formula is: if the qth task of job type l has been distributed to server r for execution in the past, then the actual benefit obtained after its execution in the past is The average value of The estimate is , otherwise it is estimated as zero. The variance of is estimated using the following formula:

[0065]

[0066] Where g(t) is a function proportional to ln(ln(t)).

[0067] Convert the original problem into a deterministic optimization problem. According to the above expectation learning framework, convert the random optimization problem given in step 1 into a deterministic problem in each time slice t. Specifically, for each time slice t∈T, introduce the following problem:

[0068]

[0069] st (1),

[0070]

[0071] Where δ(t) is a sequence that tends to 0, for example (ln(ln(t))) -1 , at each time slice t, we only need to give the optimal solution to the above problem and update the estimates of expectation and variance, which can theoretically guarantee that when T→∞, the asymptotically optimal solution to the original scheduling problem can be obtained.

[0072] The new problem introduced in step (2) is NP-hard. In order to solve it effectively, a genetic algorithm is used in step (3). Considering that this problem needs to be solved in each time slice t, t is removed from each formula in the following explanation. Specifically, it will be organized into the following stages:

[0073] Initialization phase: Encode the independent variable x of the multi-server job scheduling problem into a chromosome, generate multiple chromosomes to form a population, and set the method termination condition. As mentioned above, the meaning of each element of x is: whether the qth task of job type l is distributed to server r for execution, and the value of each element is selected from the set {0, 1}.

[0074] When each chromosome x is initialized, it will check whether the constraint condition (1) is satisfied. If not, it will be reinitialized. Then, the maximum number of iterations of the algorithm is set to NI, and the initial value of the variable "global target maximum value" is set to 0.

[0075] Mutation phase: For each chromosome x, randomly select a job type l, and for each of its tasks q∈Q l , select an r′ from any server connected to it and Set to 1, then check whether constraint condition (1) is satisfied. If not, perform mutation again.

[0076] Crossover phase: Select two chromosomes x1 and x2 according to the following probability distribution:

[0077]

[0078] A server r is randomly selected, and the job tasks distributed to server r on the two selected chromosomes x1 and x2 are exchanged to form two new chromosomes.

[0079] Selection phase: optimize the objective function among the current P chromosomes The chromosome with the largest value is selected. If the chromosome makes the optimization objective function value larger than the global maximum objective value, the "global maximum objective value" is updated to this value, and the scheduling plan corresponding to the chromosome is recorded as the final result of the optimization.

[0080] Repeat the steps from the mutation phase to the selection phase until the termination condition is met. The termination condition is: the maximum number of cycles set in the initialization phase, and the service selection scheme corresponding to the chromosome with the highest fitness is selected as the final result.

[0081] In summary: the proposed scheduling strategy aims to maximize the group economic benefits rather than the job completion time. The proposed scheduling algorithm does not require the running time of a single job (or a single task of a job) on different servers to be known, but estimates its running time through historical data. Considering that the task will be affected by various factors during execution (the process management strategy of the operating system, the workload of the existing load, whether frequency reduction occurs, etc.), its running time on any server is bound to fluctuate and cannot be known in advance. However, experience shows that the running time usually obeys an unknown distribution. The proposed strategy will learn the expectation and variance of the unknown distribution based on historical data, so as to make better scheduling strategies and maximize economic benefits.

[0082] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-server operation optimization scheduling method, characterized in that: The following steps are involved: 1) A general bipartite graph-based group scheduling model is proposed, with the optimization goal of maximizing Asw, where Asw represents social benefits. MSJ includes parallel deep learning training task composition and parallel deep learning training workflow. MSJ represents multi-server jobs. A general heterogeneous server cluster R has multiple servers in the cluster. r∈R represents a server. Consider K different types of computing devices. Each server is installed with different types and numbers of computing devices. For server r, the number of the kth computing device it is equipped with is c (k,r) , using the vector c = [c (k,r) ] k∈K,r∈R Represents the number of various computing devices installed on each server in the cluster. The scheduling model is modeled as a bipartite graph (L, R, E). In the bipartite graph (L, R, E), L is the set of types of multi-server jobs, R is the set of heterogeneous servers in the cluster, and E is the edge in the bipartite graph. Its physical meaning is: any edge e = (l, r) ∈ E exists if and only if the l-th type of job can be distributed to the r-th server for execution. The job type is called a "port" to wait for the job to arrive, and the edge is called a "channel", which is the path where a single task of each job can be distributed. An edge e = (l, r) ∈ E exists if and only if the following two conditions are met at the same time: server r meets the node affinity of job type l; the number of computing devices installed on server r meets the requirements of job type l; 2) Based on this model, a framework is proposed to learn the expectation and variance of MSJ running time on different servers through historical reward data. The distribution of the running rate of the qth task of job type l on server r is recorded as The expected value of this distribution is recorded as Continuously estimate the expected value through the specific returns of historical time slices And the variance is estimated. The scheduling for MSJ is called group scheduling. According to the different input data, the scheduling problem is divided into the following two categories: scheduling of batch jobs and scheduling of streaming jobs; 3) Based on this framework, a genetic algorithm-based Asw maximization strategy is proposed, which can maximize the overall social benefits while ensuring that various types of constraints are met, including: Initialization phase: Encode the independent variable x of the multi-server job scheduling problem into a chromosome, generate multiple chromosomes to form a population, set the method termination condition, and check whether the constraint condition (1) is satisfied when each chromosome x is initialized. If not, reinitialize it, then set the maximum number of iterations of the algorithm to NI, and set the initial value of the variable global target maximum value to 0; Mutation phase: For each chromosome x, randomly select a job type l, and for each of its tasks q∈Q l , select r' from any server connected to it and Set to 1, and then check whether constraint condition (1) is satisfied. If not, mutate again. The constraint condition (1) is: in each time slice t, for each k∈K, r∈R, the scheduling decision x(t) needs to satisfy the following computing resource constraints: The number of tasks of job type l is Q l , for any task q∈Q l , let the number of its demands for the kth resource be At each time slice t, record is the scheduling decision variable, where q∈Q l , which means whether the qth task of job type l is distributed to server r for execution, using vector Indicates that all scheduling variables need to be decided within time slice t. For server r, the number of k-th computing devices installed on it is c (k,r) ; Crossover phase: Select two chromosomes x1 and x2 according to the following probability distribution: It is obtained based on the estimated expectation and variance of the running time on different servers; Randomly select a server r, and exchange the jobs distributed to server r on the two selected chromosomes x1 and x2 to form two new chromosomes; Selection phase: optimize the objective function among the current P chromosomes The chromosome with the largest value is selected. If the chromosome makes the optimization objective function value larger than the global maximum objective value, the global maximum objective value is updated to this value, and the scheduling plan corresponding to the chromosome is recorded as the final result of the optimization.

2. The method for optimizing and scheduling multi-server operations according to claim 1, characterized in that: The parallel deep learning training task group consists of: a parameter server, which is a process running on a physical machine or a virtual machine, referred to as PS; A plurality of worker nodes, each of which is a process for performing back propagation.

3. The method for optimizing and scheduling multi-server operations according to claim 1, characterized in that: The workflow of the parallel deep learning training is as follows: in each round of iteration, each Worker updates its local gradient through the back propagation algorithm according to its allocated local data set, and then transmits the local gradient to the PS through the network.

4. The method for optimizing and scheduling multi-server operations according to claim 3, characterized in that: After the PS collects all local gradients, it obtains the global gradient of this round by taking the average value. The PS transmits the global gradient to each Worker through the network to enter the next round of iteration.

5. The method for optimizing and scheduling multi-server operations according to claim 1, characterized in that: The batch processing job refers to a task that only requires a single input and automatically releases resources after the task is completed. After these tasks arrive at different times, the scheduler needs to specify a server that can run each task and allocate computing resources to the task. When specifying a server, the scheduler can only select a server that can meet the resource requirements.

6. The method for optimizing and scheduling multi-server operations according to claim 1, characterized in that: The streaming job refers to a job in which different data is input in different time windows to perform corresponding calculations. Typical streaming jobs are real-time big data analysis. These jobs must always be running and need to process real-time input data. At this time, what needs to be scheduled is the input data stream.

Images