Cluster job scheduling method and system

Abstract

The invention discloses a cluster job scheduling method and a cluster job scheduling system, which are used for modeling a computing job submitted by a user into a directed acyclic graph (DAG) so as to convert a real-time scheduling problem into an allocation problem between resources and the DAG. The method comprises a normal state scheduling mode and a special state scheduling mode which are respectively used for realizing an efficiency target and a fair target. Wherein the normal scheduling mode is based on a graph neural network, job information is collected from local to overall, the information is input to the reinforcement learning model, and a scheduling decision for minimizing average job completion time is given; the special state scheduling mode detects the long-tail job in real time according to a statistical rule of job execution time, performs special optimization for the long-tail job, and preferentially allocates resources for the long-tail job to reduce the completion time of the long-tail job. The scheduling system is switched between the two scheduling modes, and the balance between efficiency and fairness is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster task scheduling and reinforcement learning, and in particular to a cluster job scheduling method and system. Background Art

[0002] In recent years, with the popularization of mobile devices, the improvement of software ecology and the continuous advancement of Internet communication technology, society has entered the era of big data. In this era, computing clusters integrate and share the resources of multiple computing devices, transmit information between devices through communication networks, and strive to achieve reasonable resource allocation, thereby improving the execution efficiency of computing tasks. In fact, computing clusters have become one of the important methods for processing computing tasks in the era of big data.

[0003] In computing clusters, the scheduler plays an important role in managing and distributing system resources and scheduling user job execution. Depending on whether available information is extracted from the job, scheduling strategies can be divided into two categories: traditional static strategies and content-aware dynamic strategies.

[0004] The first category is the traditional static scheduling strategy, including first-come-first-served (i.e., allocating resources in the order of arrival at the cluster), shortest job first (i.e., jobs with shorter expected execution time will be scheduled first), fair scheduling (i.e., all jobs share all resources equally), etc. Their characteristics are that they do not need to consider the specific data distribution and structure, but use pre-established rules to deal with different jobs. The huge benefit brought by this is that the cost of policy formulation is low, which reduces the workload of modification and debugging. However, the generalization is not high enough. When resources are limited and there are many user requests, it will inevitably affect the overall performance of the system. For example, if the shortest job first scheduling strategy is adopted, after a long job arrives, if the system continues to receive requests for some shorter jobs, this will prevent the long job from obtaining resources for a long time.

[0005] The second category is content-aware dynamic scheduling strategies. This approach extracts useful information from jobs, such as execution phase characteristics, data dependencies, etc., to help formulate strategies that meet high-level optimization goals. By understanding the information of each computing task itself, the scheduler can take a more macro perspective, comprehensively consider the resources and job conditions in the current system, focus on the overall system, make full use of limited resources, and provide a scheduling method that is close to the optimal solution. Although this strategy can achieve good results, if manual fine-tuning is used, it requires a lot of domain knowledge and time and energy, is difficult, and generalization is difficult to guarantee.

[0006] According to the analysis of existing research on cluster job scheduling by the current inventors, the following problems still exist: (1) Resource limitation on the cluster. Due to the increase in data scale, there is also resource limitation in the computing cluster, that is, the available resources in the current system cannot meet the needs of all computing tasks at the same time. At this time, the contradiction between resources and needs makes it a challenge to complete the real-time scheduling of data processing jobs on the distributed computing cluster. (2) Efficiency requirements of scheduling. Every user hopes that his computing request will be responded to quickly, which requires a large throughput of the system. Therefore, the average completion time of the job needs to be as small as possible, and the user's request should not be kept waiting for too long. (3) Fairness requirements of scheduling. During some busy periods, the computing cluster may not only receive a large number of user requests with different structures and different needs in a short period of time, but also due to resource limitations, it is difficult for the system to reasonably allocate available resources while ensuring fairness as much as possible, which may cause some jobs to be in a state of waiting for scheduling, or even fall into "starvation" due to the emergence of jobs with higher priority, which will seriously affect the user experience of some users.

[0007] Therefore, there is an urgent need for a scheduling strategy that balances efficiency and fairness on resource-constrained computing clusters. Summary of the invention

[0008] The purpose of the present invention is to provide a cluster job scheduling method and system, which can optimize jobs with longer execution time while minimizing the average completion time of jobs and maximizing the system throughput, so as to achieve a balance between efficiency and fairness.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions: a cluster job scheduling method, including two scheduling modes: normal scheduling and special scheduling. The normal scheduling mode is used for long-term real-time processing of computing job information to be scheduled and giving scheduling execution decisions. The special scheduling mode is used for optimizing long-tail jobs.

[0010] The normal scheduling mode includes:

[0011] The information representation module represents the computing job in the form of a directed acyclic graph (DAG). Each node of the DAG corresponds to a stage of the job, and each directed edge corresponds to the dependency between stages.

[0012] The information extraction module uses the directed acyclic graph (DAG) structure of the computing job to extract the corresponding graph embedding information, wherein the graph embedding information includes three dimensions, one dimension is the single node embedding information representing the information of each node itself and all its child nodes, one dimension is the single job embedding information representing the overall information of each job, and another dimension is the global embedding information representing the overall information between various jobs;

[0013] The scheduling decision module is used to use the reinforcement learning model to extract the graph embedding information and give a scheduling decision that minimizes the average job completion time according to the global and local job information, and submit the scheduling decision to the cluster in the form of a tuple, which includes the job nodes to be scheduled for execution and the corresponding parallel limits.

[0014] Among them, the normal scheduling mode is also used to efficiently divide resources when system resources are insufficient, and concentrate resources to complete jobs with earlier arrival times. By virtualizing the actual information in the system, the normal scheduling mode only needs to consider at most N jobs, and utilizes the information extraction and scheduling decision module for efficient execution, thereby ensuring that the cluster system minimizes the average job completion time as much as possible in resource-constrained scenarios.

[0015] The special state scheduling mode includes:

[0016] The long job detection module is used to approximately estimate the execution time of the job according to the statistical law of the expected execution time of the job, detect whether the system has a long-tail job according to the approximately estimated execution time of the job, and switch the system from the normal scheduling mode to the special state scheduling mode after detecting the long-tail job. Specifically, it includes: summing up the expected execution time of each stage carried by the job itself, taking the result as the expected execution time of the job, and comparing it with the set long-tail job execution time threshold T. Once the threshold is exceeded, the job is judged as a long-tail job, and the job is switched to the special state scheduling, and the long-tail job is scheduled for execution first;

[0017] The resource partitioning module is used to dynamically partition available resources and allocate resources to long-tail jobs based on the actual available resources in the system after detecting long-tail jobs.

[0018] Long job priority scheduling module: used to prioritize the long-tail jobs detected and switch the system from the special scheduling mode to the normal scheduling mode after the long-tail jobs are completed.

[0019] Among them, the approximate estimation of the execution time of the job by the long job detection module is based on the distribution of the execution time of a large number of jobs in the actual system. In terms of statistical laws, the number of long-tail jobs is small and the execution time is long, so the approximate estimation can be used to detect long-tail jobs. The long-tail jobs refer to those jobs that appear occasionally but have very long execution times, such as jobs that exceed the 99th percentile JCT. The data scale to be queried is generally large, and the actual completion time is also the longest. If the completion time of the long-tail job differs too much from the average JCT, it means that the corresponding scheduling execution may ignore fairness.

[0020] The special state scheduling mode is used to optimize the completion time of long-tail jobs to ensure fairness. Two indicators are used to measure fairness, namely the standard deviation of the completion time of all jobs and the ratio of the completion time of long-tail jobs to the average JCT. The smaller the standard deviation and the ratio, the smaller the distribution difference of the completion time between jobs, that is, the better the fairness.

[0021] This scheduling method is suitable for job scheduling in resource-constrained systems, and fully utilizes resources such as computing and storage by switching between normal scheduling mode and special scheduling mode, taking into account both efficiency and fairness. Resource-constrained systems are used to represent situations in which computing and storage resources cannot meet the needs of all jobs at the same time in actual production environments. Among them, the normal scheduling mode is used to reduce the average completion time of all jobs to achieve efficiency goals; the special scheduling mode is used to reduce the completion time of long-tail jobs to achieve fairness goals.

[0022] According to the above technical solution, each node of the DAG in the information representation module of the normal scheduling mode contains a serial task sequence, and the DAG representation of each job contains the estimated execution time of each task, which is used to provide a basis for the approximate estimation of the long job detection module of the special scheduling mode;

[0023] The information representation module of the normal scheduling mode also provides partial attribute information of each node itself, and the partial attribute information includes but is not limited to the number of tasks to be executed in the node and the expected execution time of each task.

[0024] According to the above technical solution, the graph embedding information includes single-node embedding information, single-job embedding information and global embedding information;

[0025] A single node embedding stores information from that node and all its child nodes; a single job embedding aggregates information from the entire DAG; on this basis, combined with the embedding information of each job, all cluster information is obtained and integrated into a global embedding. How to select the information stored in these embedding vectors is not hard-coded, but is determined by the system itself during the learning process.

[0026] Single node embedding information is used to describe the aggregate information of a node itself and all its child nodes in the DAG. The implementation adopts a bottom-up access order, starting from the leaf node of the DAG and calculating layer by layer. In each step of the message passing process of the graph neural network, if all the child nodes of the node v to be calculated have completed the information integration, the embedding information of the node v is calculated according to the embedding vectors of all the child nodes;

[0027] Single job embedding information is used to describe the overall information of a job corresponding to the DAG. An additional sink t is added to each DAG, and a directed edge is added between each original node and t. That is, each original node is a child of t. The sink t is used to collect and aggregate the information of each node. The embedded information of each job is represented by y. i To express;

[0028] The global embedding information is used to describe the overall information reflected by the DAG corresponding to all jobs in the current system. In addition to the sink t added by the single job embedding information, an additional super sink r is added, and a directed edge is added between each sink t and r, that is, each sink t is a child node of r. The super sink r is used to collect and aggregate the information of each job, represented by z.

[0029] According to the above technical solution, assuming is the directed acyclic graph G corresponding to the i-th task i The original attribute vector of node v in the matrix, the embedding vector of node v can be obtained according to the following formula:

[0030]

[0031] in, represents the embedding vector of node v, f1(.) and g1[.] are both nonlinear transformation functions for vectors, implemented by neural networks; ξ(v) represents the set of child nodes of node v; g1[.] represents the summary of the embedding information of child nodes, Represents the properties of node v;

[0032] The single job embedding vector calculation formula is: in represents the embedding vector of node u in job i, f2(.) and g2[.] are nonlinear transformation functions for vectors;

[0033] The global embedding vector calculation formula is: z = g3[∑ i f3(y i )], where y i represents the single job embedding vector of job i, f3(.) and g3[.] are nonlinear transformation functions for the vector.

[0034] According to the above technical solution, the execution steps of the reinforcement learning model include:

[0035] The scheduling decision is encoded as a tuple (s, limit), where s represents the next stage to be scheduled, that is, the node in the corresponding DAG; limit represents the upper limit of the resources that can be used by the job in s;

[0036] Select the next stage s to be scheduled, use the graph embedding information in the three dimensions that have been calculated, and use the scoring function q to calculate the scheduling score of each currently schedulable node v The scoring function q is implemented by a neural network and is used to map the embedding vector to a scalar value. It accepts three parameters as input, which are the single node embedding information of stage v. The single job embedding information y of the job in stage v i and global embedded information z, output the corresponding scheduling score; scheduling score It is used to indicate the priority of the next step of scheduling v. The larger the score, the greater the probability of being selected for scheduling.

[0037] After calculating the scheduling scores of all nodes v that can be scheduled for execution, the probability of node v being the next scheduling node is calculated through the softmax operation, and the node with the largest probability s is selected for scheduling execution;

[0038] Select the parallel limit and use the scoring function w to calculate the parallelism score of the resource limit l of each job. The scoring function w is implemented by a neural network and is used to map the embedding vector to a scalar value. It accepts three parameters as input, which are the single job embedding information y of the corresponding job. i , globally embed information z and parallelism l, and output the corresponding parallelism score; the parallelism score is used to indicate the priority of allocating the number of resources to a job. The larger the score, the greater the probability of allocating the corresponding number of resources;

[0039] After calculating the parallelism scores of each job, the probability of selecting the parallelism is calculated through the softmax operation, and the job i where the job is located is found according to the selected scheduling execution node s, and the resource limit limit with the highest probability for job i is selected.

[0040] According to the above technical solution, the probability calculation formula of the node v as the next scheduling node is:

[0041]

[0042] Where P(node=v) represents the probability of node v being the next scheduling node, ζ t is the set of all currently schedulable nodes, represents the scheduling score of node v in job i, represents the scheduling score of the schedulable node u, and j(u) is the job where node u is located.

[0043] According to the above technical solution, the reinforcement learning model uses the policy gradient algorithm to perform gradient descent updates on the parameters of the neural network, and both the graph neural network and the policy network are differentiable; during the training process, the reinforcement learning agent updates its own parameters as follows:

[0044]

[0045] Among them, α is the learning rate, T is the number of decisions, and π θ (s k ,a k ) means that when the parameter is θ, in state s k Take action k The probability of k represents a decision round, s k Indicates the state of the round, a k Indicates the action taken in this round, represents the gradient about θ, r k′ represents the reward for each action taken in the future, b k is a baseline value used to reduce policy gradient differences.

[0046] In order to learn a comprehensive and reliable scheduling strategy, the reinforcement learning model uses continuously arriving jobs for training. However, since the parameters of the policy network in the early stage of training are randomly generated, the strategy given by the reinforcement learning agent at this time lacks generalization, does not have good results, and cannot be efficiently scheduled. The idea of ​​curriculum learning is used to continuously improve the generalization of the model and save training time. Specifically, the first few rounds of training must be terminated early, and the training length is continuously extended in subsequent training rounds, so that the parameters of the reinforcement learning agent are slowly tuned, thereby gradually adapting to increasingly complex scheduling situations.

[0047] According to the above technical solution, the resource partitioning module of the special scheduling mode dynamically partitions the existing available resources on the cluster, and the maximum number of resources that each job can obtain is the same. The resource partitioning module preferentially allocates resources to long jobs until the long job reaches the resource limit or there are no available resources in the system. The specific execution steps include:

[0048] 1) Assuming that the total amount of resources is n, the current amount of available resources is f, and the number of unfinished jobs is U, then following the design principle of fairness, the maximum amount of resources that can be allocated to each job is:

[0049]

[0050] 2) Allocate resources preferentially for each long-tail job in the system. Assume that the amount of resources already allocated to a certain long-tail job is cur. If cur < limit, it means that the long-tail job can continue to be allocated resources, and the updated amount of resources allocated to this long-tail job is:

[0051] cur ← min(limit, f);

[0052] The above formula means that the long-tail job reaches the upper limit of allocable resources limit, or obtains all available resources f. The function min(.) represents taking the minimum of the two;

[0053] 3) If there are still remaining available resources after the long-tail job is allocated resources, then allocate the remaining available resources to other jobs;

[0054] 4) Repeat steps 2) and 3) until the resource allocation is completed.

[0055] According to the above technical solution, the long-job priority scheduling module of the special scheduling mode is also used to encode the policy given by the resource partitioning module into the form of a binary tuple, so as to be compatible with the scheduling decision module of the normal scheduling mode, and is used to improve the stability and compatibility of the system operation.

[0056] There is also another embodiment. The cluster job scheduling system models the computing jobs submitted by users as a directed acyclic graph DAG, thereby transforming the real-time scheduling problem of cluster jobs into an allocation problem between resources and DAG, and realizes the real-time scheduling of cluster jobs by allocating resources to the DAG according to a cluster job scheduling method of the above technical solution.

[0057] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention fully considers the resource-limited situation, efficiency requirements and fairness guarantee of cluster scheduling. The present invention can both use the efficient policy given by the reinforcement learning model to minimize the average job completion time as much as possible most of the time (normal scheduling mode), and can use the optimization algorithm of resource dynamic partitioning to accelerate the completion speed of long-tail jobs when detecting long-tail jobs (special scheduling mode), so as to balance efficiency and fairness. The present invention greatly reduces the completion time of long-tail jobs while slightly increasing the average job completion time, and greatly improves fairness while maintaining efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0059] Figure 1 It is an operation mode diagram of a cluster job scheduling system of the present invention;

[0060] Figure 2 is a flow chart of a normal dispatch mode of an embodiment;

[0061] Figure 3 Schematic diagram of extracting DAG single node embedded information in the normal scheduling mode of the embodiment;

[0062] Figure 4 is a schematic diagram of extracting DAG single job embedding information and global embedding information in the normal scheduling mode of the embodiment;

[0063] Figure 5 is an execution flow chart of virtualization of actual operation information of an embodiment;

[0064] Figure 6 It is a comparison chart of the present invention and other scheduling strategies on average JCT;

[0065] Figure 7 It is a comparison chart of the standard deviation of the average JCT between the present invention and other scheduling strategies;

[0066] Figure 8 It is a comparison chart of the ratio of average JCT to long-tail JCT between the present invention and other scheduling strategies. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Embodiment 1 A cluster job scheduling system is a scheduling system running on a large-scale computing cluster, which models the computing jobs submitted by users as directed acyclic graphs (DAGs), thereby converting the real-time scheduling problem into an allocation problem between resources and DAGs. The system includes two scheduling modes, normal and special, which are used to achieve efficiency goals and fairness goals respectively. Among them, the normal scheduling mode is based on a graph neural network, which collects job information from local to global, and inputs this information into a reinforcement learning model to minimize the average job completion time (JobCompletionTime, JCT); the special state scheduling mode detects long-tail jobs in real time according to the statistical laws of job execution time, and performs special optimization for long-tail jobs, completing them as soon as possible to reduce the completion time of long-tail jobs. The scheduling system switches between the two scheduling modes, weighing efficiency and fairness.

[0069] An operation mode of a cluster job scheduling system is as follows: Figure 1 As shown in the figure, it accepts job requests and system resource information externally and outputs scheduling strategies; it runs on two scheduling modes internally. The left side is the normal scheduling mode, which includes: information representation module, information extraction module and scheduling decision module.

[0070] The information representation module is used to model and maintain the information of each job in the structure of a directed acyclic graph (DAG). Each DAG corresponds to a computing job, and each node in the DAG corresponds to a stage of the computing job. Each directed edge from node i to node j in the DAG corresponds to the stage j in the computing job that must be executed after stage i is completed. Each node in the DAG also has a task sequence that is executed serially and carries relevant attribute values.

[0071] Based on the information representation module, the information extraction module uses a graph convolutional neural network to encode the state information in the DAG, and the encoded information is called an embedding vector. The graph embedding process takes the DAG corresponding to all jobs as input, and the output includes three embedding vectors, namely single node embedding, single job embedding and global embedding, which are used to represent information at the node, job and global levels, that is, to collect the graph embedding information of jobs in the DAG from three dimensions.

[0072] The scheduling decision module uses the reinforcement learning (RL) model to infer the DAG information obtained by the information extraction module to obtain a decision that is as efficient as possible. Specifically, in order to avoid the increase in system complexity caused by an excessively large action space and an excessively long action sequence, and to enable normal training of reinforcement learning, a two-dimensional action is used to represent a decision in the form of (s, limit). Among them, s is the next stage to be scheduled, and limit is the upper limit of the amount of resources that can be used by the job in s. If there are T actions in a round of reinforcement learning (epi sode), a reward value r will be obtained based on the effect of the kth action. k When the high-level scheduling goal is to minimize the average JCT, r k =-(t k -t k-1 )J k , where J k It means that in [t k-1 ,t k ) in the time range.

[0073] The right side is the special scheduling mode, which includes a long job detection module, a resource partitioning module, and a long job priority scheduling module.

[0074] Among them, the long job detection module is used to approximately estimate the execution time of the job according to the statistical law of the expected execution time of the job, detect whether the system has a long-tail job according to the approximately estimated execution time of the job, and switch the system from the normal scheduling mode to the special state scheduling mode after detecting the long-tail job, specifically including: summing up the expected execution time of each stage carried by the job itself, taking the result as the expected execution time of the job, and comparing it with the set long-tail job execution time threshold T. Once the threshold is exceeded, the job is judged as a long-tail job, and the job is switched to the special state scheduling, and the long-tail job is scheduled for execution first;

[0075] The resource partitioning module is used to dynamically partition available resources into long-tail jobs and allocate storage resources according to the actual available resources in the system after detecting the long-tail jobs.

[0076] Long job priority scheduling module: used to prioritize the long-tail jobs detected and switch the system from the special scheduling mode to the normal scheduling mode after the long-tail jobs are completed.

[0077] Embodiment 2 A cluster job scheduling method includes two scheduling modes: normal scheduling and special scheduling. The normal scheduling mode is used for long-term real-time processing of computing job information to be scheduled and giving scheduling execution decisions. The special scheduling mode is used for optimizing long-tail jobs.

[0078] Among them, the information representation module represents the computing job in the form of a directed acyclic graph (DAG). Each node of the DAG corresponds to a stage of the job, and each directed edge corresponds to the dependency relationship between stages. Each node of the DAG contains a serial task sequence, and the DAG representation of each job includes the estimated execution time of each task. Each node in the DAG also has a serially executed task sequence that carries relevant attribute values.

[0079] The information extraction module uses a graph convolutional neural network to extract the corresponding graph embedding information from the directed acyclic graph (DAG) structure of the computing job, and encodes the extracted graph embedding information of the DAG. The encoded information is called an embedding vector. The graph embedding process takes the DAG corresponding to all jobs as input, and the output includes three embedding vectors, namely single node embedding, single job embedding and global embedding, which are used to represent information at the node, job and global levels.

[0080] Single-node embedding information is used to describe the aggregate information of a node itself and all its child nodes in the DAG. It is implemented in a bottom-up access order, starting from the leaf node of the DAG and calculating layer by layer. In each step of the message passing process of the graph neural network, all child nodes of the node v to be calculated have completed the information integration. The embedding vectors of all child nodes can be directly used to calculate the embedding information of the node v, thereby realizing the upward transmission of information. The formal description is as follows:

[0081] Assumptions is the directed acyclic graph G corresponding to the i-th task i The original attribute vector of node v in the matrix, the embedding vector of node v can be obtained according to the following formula:

[0082]

[0083] Wherein, f and g are nonlinear transformation functions for vectors, which are implemented by neural networks and use different nonlinear transformation functions in the three embedding dimensions; ξ(v) represents the set of child nodes of node v; the first term on the right side of the equal sign is a summary of the embedding information of the child nodes, which is consistent with the second term of the node v’s own attributes. Adding them together, we get the embedding of v itself;

[0084] For the single job embedding information, an additional sink t is added to each DAG, and a directed edge is added between each original node and t, that is, each original node is a child of t. The sink t is used to collect and aggregate the information of each node. The embedding information of each job is represented by y. i To express;

[0085] The global embedding information is used to describe the overall information reflected by the DAG corresponding to all jobs in the current system. In addition to the sink t added by the single job embedding information, an additional super sink r is added, and a directed edge is added between each sink t and r, that is, each sink t is a child node of r. The super sink r is used to collect and aggregate the information of each job, represented by z.

[0086] The scheduling decision module is used to use the reinforcement learning model to extract the graph embedding information and give a scheduling decision that minimizes the average job completion time based on the global and local job information. The scheduling decision is submitted to the cluster in the form of a binary. The specific execution steps include:

[0087] A two-dimensional action is used to represent a decision, in the form of (s, limit). s is the next stage to be scheduled, and limit is the upper limit of the amount of resources that can be used by the job in s. If there are T actions in a round (episode) of reinforcement learning, a reward value r will be obtained based on the effect of the kth action. kWhen the high-level scheduling goal is to minimize the average JCT, r k =-(t k -t k-1 )J k , where J k It means that in [t k-1 ,t k ) is the number of jobs in the system within the time range. The goal of the reinforcement learning algorithm is to minimize the expectation of the penalty:

[0088]

[0089] According to Little's law, minimizing the number of jobs in the system also minimizes the average JCT. The policy gradient algorithm is used to perform gradient descent updates on the parameters of the neural network, using π θ (s t ,a t ) means that when the parameter is θ, in state s t Take action t The probability of; during the training process, the reinforcement learning agent updates its own parameters as follows:

[0090]

[0091] Among them, α is the learning rate, T is the number of decisions, and π θ (s k ,a k ) means that when the parameter is θ, in state s k Take action k The probability of k represents a decision round, s k Indicates the state of the round, a k Indicates the action taken in this round, represents the gradient about θ, r k′ represents the reward for each action taken in the future, b k is a baseline value used to reduce policy gradient differences.

[0092] In order to learn a comprehensive and reliable scheduling strategy, the reinforcement learning model uses continuously arriving jobs for training. However, since the parameters of the policy network in the early stage of training are randomly generated, the strategy given by reinforcement learning at this time lacks generalization, does not have good results, and cannot be efficiently scheduled. The idea of ​​curriculum learning is used to continuously improve the generalization of the model and save training time. The specific implementation is as follows:

[0093] The first few rounds of training are terminated early, and the training length is continuously increased in subsequent rounds, so that the parameters of reinforcement learning can be slowly tuned to gradually adapt to increasingly complex scheduling situations.

[0094] Considering the limited resources, in order to make full use of resources, it is not advisable to let the reinforcement learning model schedule too many jobs at the same time, which will increase the number of waiting jobs and reduce the average JCT. Virtualize the actual job information in the system so that the normal scheduling mode only perceives a part of the jobs, thereby improving the execution efficiency of these jobs. In short, set an execution pool for the scheduling range of reinforcement learning. Jobs enter the execution pool in the order of submission. If the execution pool is full, wait for vacancies and only schedule jobs in the execution pool, thereby ensuring the fairness of the jobs that arrive first in resource-constrained scenarios. Among them, the tuple includes the job node to be scheduled for execution and the corresponding parallel limit.

[0095] Special scheduling modes include:

[0096] The long job detection module is used to estimate the execution time of the job approximately according to the statistical law of the expected execution time of the job, detect whether the system has long-tail jobs according to the approximately estimated execution time of the job, and switch the system from the normal scheduling mode to the special state scheduling mode after detecting the long-tail job. Specifically, it includes: summing the expected execution time of each stage carried by the job itself, taking the result as the expected execution time of the job, and comparing it with the set long-tail job execution time threshold T. Once the threshold is exceeded, the job is judged as a long-tail job, and it will be switched to the special state scheduling, and the long-tail job will be scheduled for execution first. Specifically, the job request received by the computing cluster usually gives the expected computing time of each task in each stage of itself. Since the resources obtained by the job and the degree of parallelism adopted cannot be accurately known in the static period, it is an intuitive idea to sum the expected computing time of all tasks in the job as an estimate of the time required for the calculation of the entire job. Under resource-constrained conditions, one of the optimization goals of the present invention is the execution time of the 99th percentile job. Through experiments, it can be verified that the overall distribution trend is still approximate, and the completion time of most jobs is concentrated in the shorter part, while the number of long jobs is small, and one trend is that the longer the completion time, the smaller the number. According to such statistical laws, it is reasonable to use the accumulated expected time of all tasks to approximate the percentile of the actual completion time. In implementation, whenever a job with an expected computing time exceeding the threshold T is received, it can be considered that a long-tail job has been found, and the system will switch from the normal scheduling mode to the special state scheduling mode to optimize the long-tail job.

[0097] A resource partitioning module, which is used to allocate more computing and storage resources to long-tail jobs according to the actual available resource situation in the system after detecting long-tail jobs; specifically, the limited available resources are preferentially allocated to long-tail jobs to accelerate the completion speed of long-tail jobs. Following the principle of fairness, by dynamically partitioning the existing available resources on the cluster, the maximum amount of resources that each job can obtain is the same, but the resource partitioning module will also preferentially allocate resources to long jobs until the long job reaches the resource limit or there is no available resource in the system. The execution steps include:

[0098] 1) Assume that the total amount of resources is n, the current available resource amount is f, and the number of unfinished jobs is U. Then, following the fairness design principle, the maximum amount of resources that each job can be allocated is:

[0099]

[0100] 2) Preferentially allocate resources to each long-tail job in the system. Assume that the amount of resources that a certain long-tail job has already received is cur. If cur < limit, it means that the long-tail job can continue to be allocated resources, and the updated amount of resources that the long-tail job continues to be allocated is:

[0101] cur ← min(limit, f);

[0102] It means that the amount of resources allocated to the long-tail job is the minimum of the resource limit of the job and the current available resources, representing the minimum value function;

[0103] 3) If there are still remaining available resources after the long-tail job is allocated resources, then allocate the remaining available resources to other jobs;

[0104] 4) Repeat steps 2) and 3) until the resource allocation is completed.

[0105] A long-job priority scheduling module: which is used to preferentially schedule the detected long-tail jobs, encode the scheduling decision in the form of a binary tuple, and according to the allocation strategy of the resource partitioning module, preferentially schedule long-tail jobs to obtain resources and execute calculations until all long-tail jobs in the system are completed. After the long-tail jobs complete their execution, the system is switched from the special-state scheduling mode to the normal-state scheduling mode.

[0106] When there are no long-tail jobs in the system, the system exits the special-state scheduling mode and enters the normal-state scheduling mode, and uses the perceived job information to make efficient scheduling decisions.

[0107] A comparative experiment is conducted on a cluster job scheduling method proposed by the present invention and a similar cluster job scheduler in the prior art. The experimental results are as Figure 6 and Figure 7As shown in Figure 2, since the system is deployed in a real-time environment, each indicator is the average result of multiple rounds of experiments. Figure 6 It is a comparison chart of average JCT, which is used to characterize the efficiency of the system scheduling strategy. The lower the average JCT, the greater the system throughput and the more efficient the scheduling strategy. Figure 7 and Figure 8 They are the comparison charts of the standard deviation of JCT and the ratio of long-tail JCT to average JCT. The lower the two are, the smaller the difference between the completion time of each job is, and the fairer the scheduling strategy is. Compared with the other two existing systems, Decima and heuristic algorithms, under the condition of limited resources, the efficiency of the scheduler proposed by the present invention is significantly better than that of the heuristic algorithm, and the fairness is significantly better than that of Decima, achieving a balance between efficiency and fairness. Among them, Decima is an efficient scheduling system using reinforcement learning.

[0108] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0109] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A cluster job scheduling method, including two scheduling modes: normal scheduling and special scheduling. The normal scheduling mode is used to process the computing job information to be scheduled in real time for a long time and give scheduling execution decisions. The special scheduling mode is used to optimize long-tail jobs, and is characterized by: The normal scheduling mode includes: The information representation module represents the computing job in the form of a directed acyclic graph (DAG). Each node of the DAG corresponds to a stage of the job, and each directed edge corresponds to the dependency between stages. The information extraction module uses the DAG structure of the computing job to extract the corresponding graph embedding information; A scheduling decision module, used to use a reinforcement learning model to extract the graph embedding information and give a scheduling decision that minimizes the average job completion time according to global and local job information, and submit the scheduling decision to the cluster in the form of a binary, where the binary includes the job node to be scheduled for execution and the corresponding parallel limit; The special state scheduling mode includes: The long job detection module is used to approximately estimate the execution time of the job according to the statistical law of the expected execution time of the job, detect whether the system has a long-tail job according to the approximately estimated execution time of the job, and switch the system from the normal scheduling mode to the special state scheduling mode after detecting the long-tail job. Specifically, it includes: summing up the expected execution time of each stage carried by the job itself, taking the result as the expected execution time of the job, and comparing it with the set long-tail job execution time threshold T. Once the threshold is exceeded, the job is judged as a long-tail job, and the job is switched to the special state scheduling, and the long-tail job is scheduled for execution first; The resource partitioning module is used to dynamically partition available resources and allocate resources to long-tail jobs based on the actual available resources in the system after detecting long-tail jobs. Long job priority scheduling module: used to prioritize the long-tail jobs detected and switch the system from the special scheduling mode to the normal scheduling mode after the long-tail jobs are completed.

2. A cluster job scheduling method according to claim 1, characterized in that: Each node of the DAG in the information representation module of the normal scheduling mode contains a serial task sequence, and the DAG representation of each job contains the estimated execution time of each task; The information representation module of the normal scheduling mode also provides partial attribute information of each node itself, and the partial attribute information includes but is not limited to the number of tasks to be executed in the node and the expected execution time of each task.

3. A cluster job scheduling method according to claim 1, characterized in that: The graph embedding information includes single-node embedding information, single-job embedding information and global embedding information; Single node embedding information is used to describe the aggregate information of a node itself and all its child nodes in the DAG. The implementation adopts a bottom-up access order, starting from the leaf node of the DAG and calculating layer by layer. In each step of the message passing process of the graph neural network, if all the child nodes of the node v to be calculated have completed the information integration, the embedding information of the node v is calculated according to the embedding vectors of all the child nodes; Single-job embedding information is used to describe the overall information of the DAG corresponding to a certain job. An additional sink node t is added to each DAG, and a directed edge is added between each original node and t to collect and aggregate the information of each node in the DAG; Global embedding information is used to describe the overall information reflected by the DAGs corresponding to all jobs in the current system. Outside the sink node t added in the single-job embedding information, an additional super sink node r is added, and a directed edge is added between each sink node t and r to collect and aggregate the information of each job in the system.

4. A cluster job scheduling method according to claim 3, characterized in that: Assumptions is the directed acyclic graph G corresponding to the i-th task i The original attribute vector of node v, the embedding vector formula of node v is: in, represents the embedding vector of node v in job i. Both f1(.) and g1[.] are nonlinear transformation functions for vectors, implemented by neural networks. ξ(v) represents the set of child nodes of node v. g1[.] represents the summary of the embedding information of child nodes. Represents the properties of node v; The calculation formula for the single-job embedding vector: in, represents the embedding vector of node u in job i, f2(.) and g2[.] are nonlinear transformation functions for vectors; The calculation formula for the global embedding vector: <h2 style=";text-align:left;direction:ltr">z = g3[∑<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> f3(y<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> )]; Among them, y i represents the single job embedding vector of job i, f3(.) and g3[.] are nonlinear transformation functions for the vector.

5. A cluster job scheduling method according to claim 1, characterized in that: The execution steps of the reinforcement learning model include: Encode the scheduling decision as a binary tuple (s, limit), where s represents the next stage to be scheduled, that is, the node in the corresponding DAG; limit represents the upper limit of the resources that the job where s is located can use; Select the next stage s to be scheduled, take the calculated graph embedding information as input, and use the scoring function q to calculate the scheduling score of each currently schedulable node v Scheduling score It is used to indicate the priority of the next step of scheduling v. The larger the score, the greater the probability of being selected for scheduling. After calculating the scheduling scores of all schedulable and executable nodes v, through the softmax operation, calculate the probability that node v is the next scheduling node, and select the one with the highest probability s for scheduling and execution; Select the parallelism limit. Use the calculated graph embedding information as input, and use the scoring function w to calculate the parallelism score of the resource limit l of each job. The parallelism score is used to represent the priority of allocating the number of resources to a certain job. The larger the score, the greater the probability of allocating the corresponding number of resources; After calculating the parallelism scores of each job, through the softmax operation, calculate the probability of selecting this parallelism, and according to the already selected scheduling and execution node s, find the job i where it is located, and select the resource limit limit with the highest probability for job i.

6. A cluster job scheduling method according to claim 5, characterized in that The calculation formula for the probability that node v is the next scheduling node: Where P(node=v) represents the probability of node v being the next scheduling node, ζ t is the set of all currently schedulable nodes, represents the scheduling score of node v in job i, represents the scheduling score of the schedulable node u, and j(u) is the job where node u is located.

7. A cluster job scheduling method according to claim 5, characterized in that: The reinforcement learning model uses the policy gradient algorithm to perform gradient descent updates on the parameters of the neural network, and both the graph neural network and the policy network are differentiable; during the training process, the process of the reinforcement learning agent updating its own parameters θ is as follows: Among them, α is the learning rate, T is the number of decisions, and π θ (s k ,a k ) means that when the parameter is θ, in state s k Take action k The probability of k represents a decision round, s k Indicates the state of the round, a k Indicates the action taken in this round, represents the gradient about θ, r k′ represents the reward for each action taken in the future, b k is a baseline value used to reduce policy gradient differences.

8. A cluster job scheduling method according to claim 1, characterized in that: The resource partitioning module of the special scheduling mode dynamically partitions the existing available resources on the cluster. The maximum amount of resources that each job can obtain is the same. The resource partitioning module preferentially allocates resources to long jobs until the long jobs reach the resource upper limit or there is no available resource in the system. The specific execution steps include: 1) Assume that the total amount of resources is n, the current available resource amount is f, and the number of unfinished jobs is U. Then, following the design principle of fairness, the maximum amount of resources that each job can be allocated is: 2) Preferentially allocate resources to each long-tail job in the system. Assume that the amount of resources already allocated to a certain long-tail job is cur. If cur < limit, it means that the long-tail job can continue to be allocated resources. The updated amount of resources allocated to this long-tail job is: cur←min(limit,f) cur represents that the amount of resources allocated to the long-tail job is the minimum of the resource limit limit of this job and the current available resource f, and min(.) represents the minimum value function; 3) If there are still available resources left after allocating resources to the long-tail jobs, the remaining available resources will be allocated to other jobs; 4) Repeat steps 2) and 3) until all resources are allocated.

9. A cluster job scheduling method according to claim 1, characterized in that: The long job priority scheduling module of the special scheduling mode is also used to encode the strategy given by the resource partitioning module into the form of a (s, limit) tuple.

10. A cluster job scheduling system, characterized in that: The cluster job scheduling system models the computing jobs submitted by the user as a directed acyclic graph DAG, and implements real-time scheduling of cluster jobs by allocating resources to the DAG according to a cluster job scheduling method according to any one of claims 1 to 9.

Images