A method and device for scheduling a job

Abstract

An embodiment of the present invention provides a job scheduling method and device, including: obtaining job information of a task queue; using a policy network to schedule the task queue according to the job information to obtain a first vector, and using an expert strategy to schedule the task queue according to the job information to obtain a second vector; the first vector indicates the priority of each job scheduling in the task queue determined by the policy network, and the second vector indicates the priority of each job scheduling in the task queue determined by the expert strategy; evaluating the first vector and the second vector through a discriminator function to obtain a first estimated value; updating the policy network according to the first estimated value, and scheduling the jobs in the task queue according to the first vector.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of high performance computing technology, and in particular to a job scheduling method and device. Background Art

[0002] In recent years, deep learning has made great progress and development in a large number of different fields, such as computer vision, image recognition, natural language processing, and recommendation algorithms. In order to improve the accuracy of training results, the scale of the models used by people has continued to increase, and the amount of training data has also been continuously expanding. With the continuous expansion of training computation, in order to ensure that the training time is within an acceptable range, it is necessary to use distributed training methods.

[0003] High-performance computing clusters have the characteristics of high performance, high cost-effectiveness, easy expansion, and suitability for parallel tasks, so they are suitable for large-scale distributed training, physical simulation, and parallel computing tasks. Some existing large Internet companies have established their own graphics processing unit (GPU) or tensor processing unit (TPU) clusters and have corresponding task scheduling and cluster management methods. The existing mainstream scheduling method requires that the task must declare the amount of resources required when it is submitted, and the running tasks will not increase the running speed because of the new available resources. This may prevent the cluster resources from being used most efficiently. The cost of resources is very high, so it is very important to design a scheduler that can efficiently utilize resources. Summary of the invention

[0004] The embodiment of the present invention provides a job scheduling method and device for improving the resource utilization efficiency of a cluster.

[0005] In a first aspect, an embodiment of the present invention provides a job scheduling method, the method comprising:

[0006] Obtaining job information of a task queue; the task queue includes at least one job;

[0007] The task queue is scheduled according to the job information using a policy network to obtain a first vector, and the task queue is scheduled according to the job information using an expert policy to obtain a second vector; the first vector indicates the priority of each job scheduling in the task queue determined by the policy network, and the second vector indicates the priority of each job scheduling in the task queue determined by the expert policy;

[0008] Evaluate the first vector and the second vector by a discriminator function to obtain a first estimate, where the first estimate is used to indicate the probability that the first vector is an expert trajectory generated by the expert strategy;

[0009] The policy network is updated according to the first estimated value, and the jobs in the task queue are scheduled according to the first vector.

[0010] In a possible implementation, the job information includes the following information:

[0011] The submission time of each job in the task queue; the job duration of each job in the task queue; the number of central processing units occupied by each job in the task queue; the number of graphics processing units occupied by each job in the task queue; the memory size occupied by each job in the task queue; and the identification of each job in the task queue.

[0012] In a possible implementation manner, updating the policy network according to the first estimated value includes:

[0013] Inputting the first estimated value as a reward value into a value function network corresponding to the policy network to obtain an advantage function corresponding to the policy network;

[0014] The policy network is updated according to the advantage function.

[0015] In a possible implementation manner, scheduling the jobs in the task queue according to the first vector includes:

[0016] For the jobs in the task queue, the jobs are scheduled into the running queue in descending order of priority according to the first vector until the remaining resources are no longer sufficient to run the job with the highest priority among the remaining jobs in the task queue.

[0017] In a possible implementation, the jobs in the task queue are rescheduled every preset time or when a job in the task queue is completed.

[0018] In a second aspect, an embodiment of the present invention provides a job scheduling device, including:

[0019] An acquisition unit, used for acquiring job information of a task queue; the task queue includes at least one job;

[0020] A processing unit is used to schedule the task queue according to the job information using a policy network to obtain a first vector, and to schedule the task queue according to the job information using an expert strategy to obtain a second vector; the first vector indicates the priority of each job scheduling in the task queue determined by the policy network, and the second vector indicates the priority of each job scheduling in the task queue determined by the expert strategy; the first vector and the second vector are evaluated by a discriminator function to obtain a first estimated value, and the first estimated value is used to indicate the probability that the first vector is generated by the expert strategy; the policy network is updated according to the first estimated value, and the jobs in the task queue are scheduled according to the first vector.

[0021] In a possible implementation, the job information includes the following information:

[0022] The submission time of each job in the task queue; the job duration of each job in the task queue; the number of central processing units occupied by each job in the task queue; the number of graphics processing units occupied by each job in the task queue; the memory size occupied by each job in the task queue; and the identification of each job in the task queue.

[0023] In a possible implementation manner, the processing unit is specifically configured to:

[0024] Inputting the first estimated value as a reward value into a value function network corresponding to the policy network to obtain an advantage function corresponding to the policy network;

[0025] The policy network is updated according to the advantage function.

[0026] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method provided in the first aspect when executing the program.

[0027] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When the computer instructions are executed by a processor, the method provided in the first aspect is executed.

[0028] In a fifth aspect, an embodiment of the present invention further provides a computer program product, comprising computer executable instructions, wherein the computer executable instructions are used to enable a computer to execute the method provided in the first aspect.

[0029] Through the technical solutions in one or more of the above embodiments of the embodiments of the present invention, the embodiments of the present invention have at least the following technical effects:

[0030] In the embodiment provided by the present invention, the task queue is scheduled according to the job information using a policy network, which can save a lot of time cost for tuning the cluster priority function. In addition, the embodiment of the present invention can adapt to the distribution of job sets, avoiding the deterioration of the scheduling optimization effect due to the change of job set distribution. At the same time, the embodiment of the present invention integrates cluster characteristics and job set characteristics to generate a priority function, so that the embodiment of the present invention can automatically adapt to the resource expansion and contraction of the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 A schematic diagram of task queue scheduling provided by an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of a network architecture provided by an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of a process flow of a job scheduling method provided by an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of the structure of a job scheduling device provided by an embodiment of the present invention;

[0036] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0038] Reinforcement Learning (RL), also known as Reinforcement Learning, Evaluation Learning or Enhanced Learning, is one of the paradigms and methodologies of machine learning. It is used to describe and solve the problem of how an agent can maximize its rewards or achieve specific goals by learning strategies in the process of interacting with the environment (Sutton RS, Barto AG. Reinforcement Learning: An Introduction. Cambridge, MA: The MIT Press, 1998.). Reinforcement learning tasks can be represented as Markov decision processes. A finite state MDP is denoted by M, which is a four-tuple M =<S,A,T,R> , where S is the state set, A is the action set, T: S×A×S→[0,1] is the transition probability, and R: S×A×S→R is the reward function. For example, in a maze game, the agent randomly walks in the maze. At time t, it is in state s t (Somewhere in the maze), choose action a t (one of the above, below, left, and right), it will be t , a t ,s t+1 Transition to next state s t+1 , and then you will get a reward value R(s t , a t ,s t+1 ), generally the reward is 1 when the exit is found, and 0 otherwise.

[0039] Therefore, the goal of reinforcement learning is to learn an optimal strategy π: S→P(A), where P(A) is the probability distribution function over the action set A. Finding the optimal strategy π is to maximize the cumulative reward. where γ∈(0,1) is the discount factor.

[0040] Reinforcement learning is an artificial intelligence framework that uses interaction with the environment to learn knowledge and skills. In RL, the agent needs to receive reward signals sent by the environment to find the learning goal. However, in some real-world tasks, the reward function of the environment is difficult to define. Inverse reinforcement learning is a typical method for designing reward signals.

[0041] Currently, in the job scheduling process, jobs will arrive continuously over time. Figure 1As shown in the figure, generally, a cluster maintains an internal orderly job queue and collects specific information about the job to schedule the job at regular intervals. For example, a job queue of 128 is maintained in a high-performance cluster. Whenever a job is submitted, it enters the waiting state. If there is a vacant position in the job queue, the job will be pulled into the scheduling queue and scheduled according to the rules during the next round of job scheduling.

[0042] Figure 1 In the process, jobs are placed in a task queue, and a job can be considered as a task. When the task queue is vacant, the job enters the task queue. In an embodiment of the present invention, the jobs in the task queue can be prioritized to obtain a prioritized task queue. Then, the jobs in the task queue can be run, that is, when cluster resources are idle, the jobs are started to run according to the priority.

[0043] Generally, job queues are scheduled using heuristic methods such as first-in-first-out and small job priority. However, jobs are distinguished in terms of time and space: the duration a job needs to run can be regarded as the size of the job in the time dimension, and the nodes that the job needs to occupy can be understood as the size of the job in the space dimension. Scheduling methods that only consider the arrival time of jobs may cause some small jobs that can be completed quickly to wait for a long time; simply considering small jobs first will cause some large jobs to starve to death as small jobs arrive continuously. Scheduling based solely on the job time dimension will cause some short-duration jobs that occupy a large amount of space resources to have a higher priority, affecting the overall cluster operation efficiency; scheduling based solely on the job space dimension will cause long jobs to occupy cluster resources for a long time, reducing the cluster operation efficiency.

[0044] Therefore, in order to find a good scheduling strategy, the following problems need to be solved:

[0045] 1. How to generate appropriate scheduling strategies based on the characteristics of the job set.

[0046] In each scheduling process, the mainstream cluster resource manager, Slurm, will traverse each job in the task queue and schedule it based on rules. However, the rule-based method can only extract the job information in the current task queue, but cannot find the overall characteristics of the job over a period of time for flexible and reasonable scheduling. That is, the scheduling rules can only serve a fixed set of jobs that meet certain characteristics, and it takes a lot of time to explore and try. Among them, Slurm is an open source, highly scalable cluster management tool and job scheduling system for Linux clusters of various sizes.

[0047] 2. How to adapt to different job set characteristics.

[0048] In a cluster, the characteristics of a job set are not constant: with changes in the submitting user, the time of submitting the job, and the cluster resources, the characteristics of the job set will undergo corresponding changes that are difficult to model and confirm. Therefore, how to make the scheduling strategy adaptive to the changes in the characteristics of the cluster job set will also be an important research direction.

[0049] To this end, an embodiment of the present invention proposes a job scheduling method based on inverse reinforcement learning, which can explore the characteristics of a cluster job set and adaptively generate a corresponding scheduling method based on the characteristics of the cluster job set, thereby improving the resource utilization efficiency of the cluster and reducing the average waiting time ratio of user jobs.

[0050] At present, most of the research on using reinforcement learning methods to solve cluster job scheduling problems is to obtain a fixed function of computing job priority that can adapt to the distribution of the cluster job set being fed. A neural network is used as an approximate expression of this function. However, if only reinforcement learning methods are used, the learning goal is single, and the learned strategy needs to be changed accordingly under different job distributions. Therefore, the embodiment of the present invention uses an inverse reinforcement learning method to solve this problem.

[0051] The design of high-performance cluster job scheduling based on inverse reinforcement learning proposed in the embodiment of the present invention is based on the following two facts:

[0052] 1. The learning trajectory must be complete, because only after going through the complete scheduling process can we judge the pros and cons of this scheduling based on the average waiting time ratio, and thus learn. For reinforcement learning, this undoubtedly greatly increases the difficulty of learning. Among them, the trajectory is a series of state behaviors in the real world.

[0053] 2. Generally speaking, the scheduling strategy obtained by the reinforcement learning method can only produce a good scheduling effect on the job set that meets the job distribution of its learning. After changing the job set, the learned strategy does not completely match the job set, resulting in a decrease in the corresponding scheduling effect.

[0054] In the embodiment of the present invention, the general process of inverse reinforcement learning is as follows:

[0055] Step 1: Generate a scheduling strategy as the initial strategy;

[0056] Step 2: By comparing the differences between the interaction samples of the "experts" and your own interaction samples, learn the incentive function; specifically, in the inverse reinforcement learning training process, use some expert strategies that have been verified to have good results on the training job set to generate expert scheduling trajectories. Use the reinforcement learning policy network as a generator to generate your own scheduling trajectory, and use the discriminator function to determine which of the two trajectories is the expert trajectory and which is generated by the policy network. Using the idea of ​​confrontation, train the trajectory generated by the policy network to be closer to the trajectory generated by the expert strategy.

[0057] Step 3: Use the incentive function to perform reinforcement learning to improve the level of scheduling strategy;

[0058] Step 4: If the two strategies are not very different, you can stop learning, otherwise go back to step 2.

[0059] refer to Figure 2 , which is a schematic diagram of a network architecture applicable to an embodiment of the present invention. Figure 2 The schematic diagram of the high-performance cluster computing resource modeling architecture of the embodiment of the present invention. In the embodiment of the present invention, the cluster can be regarded as composed of multiple computing nodes, each node shares storage, and each node can have various types of computing resources: CPU computing resources, GPU computing resources, memory, and each node uses at least 100Gb / s Endpoint Detection and Response (EDR) Infiniband interconnection, so the communication overhead between nodes can be approximately ignored. When the number of jobs exceeds the length of the task queue, the remaining jobs need to wait for vacancies in the task queue to be filled.

[0060] The computing node in the embodiment of the present invention can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, as well as big data and artificial intelligence platforms.

[0061] like Figure 3 , is a flow chart of a job scheduling method provided by an embodiment of the present invention, including:

[0062] S301: Acquire job information of a task queue; the task queue includes at least one job.

[0063] In the embodiment of the present invention, during the training process of the reinforcement learning module, a job sample can be obtained by sampling from the job data set, that is, all jobs included in one interaction with the cluster, wherein the job information includes the following information:

[0064] The submission time of each job in the task queue; the job duration of each job in the task queue; the number of central processing units occupied by each job in the task queue; the number of graphics processing units occupied by each job in the task queue; the memory size occupied by each job in the task queue; the identification of each job in the task queue; the identification of the user who submitted each job in the task queue.

[0065] In the embodiment of the present invention, it is also necessary to initialize the reinforcement learning and inverse reinforcement learning network parameters, including the policy network parameters, the value function network parameters, and the discriminator function parameters.

[0066] Among them, the policy network is an artificial neural network that has not been fully trained and can be used for job scheduling. The specific training process is not limited and will not be repeated here.

[0067] In the embodiment of the present invention, information such as parameters corresponding to the expert strategy can also be obtained. The expert strategy can be a fully trained artificial neural network, which can be used for job scheduling and can obtain better scheduling results.

[0068] In the embodiment of the present invention, the information can be interacted with the environment and scheduled every 15 minutes or when a job in the task queue is completed.

[0069] S302: using a strategy network to schedule the task queue according to the job information to obtain a first vector, and using an expert strategy to schedule the task queue according to the job information to obtain a second vector.

[0070] The first vector indicates the priority of each job scheduling in the task queue determined by the policy network, and the second vector indicates the priority of each job scheduling in the task queue determined by the expert policy.

[0071] In an embodiment of the present invention, during the scheduling process, the policy network outputs priorities based on the job information in the task queue, that is, only the characteristics of the job are used to calculate the priority, ignoring the calculation of the queue order by the general neural network, thereby ensuring that the order of the jobs in the queue is independent of the final priority output.

[0072] For example, the task queue includes 128 jobs. Combined with the duration of each job, the number of CPUs occupied by each job, the number of GPUs occupied by each job, the memory size occupied by each job, and the identifier of each job, all jobs in the task queue can be represented by a (128, 5)-dimensional matrix. When scheduling, obtain the idle resource information of each node in the cluster to form an n+m-dimensional vector, that is, (the number of idle CPU nodes, the number of idle 2 CPU nodes, ..., the number of idle CPU nodes, the number of idle GPU nodes, the number of idle 2 GPU nodes, ..., the number of idle GPU nodes), where n is the maximum number of CPU cores and m is the maximum number of GPU cards. Add it to the end of each row of the job information matrix to generate a (128, 5+m+n)-dimensional matrix S, and input it into the policy network. After passing through the policy network, a 128-dimensional vector A is output, which is the first vector:

[0073] [x1, x2, x3, .., x 128 ]

[0074] where x i Represents the priority of the corresponding job in the task queue,

[0075] For high-performance cluster job scheduling problems, it is difficult to define the pros and cons of the intermediate process of scheduling. We can only evaluate and score the final result of scheduling and conduct feedback learning based on the set reward function. In the embodiment of the present invention, the inverse reinforcement learning method is used instead of manually setting the reward function. In the training process of inverse reinforcement learning, some expert strategies that have been verified to have good results on the training job set are used to generate expert scheduling trajectories. Therefore, the expert strategy can also be used to schedule the matrix S to generate the corresponding second vector A expert .

[0076] The specific generation process of the first vector and the second vector is not limited in the embodiment of the present invention.

[0077] S303: Evaluate the first vector and the second vector through a discriminator function to obtain a first estimated value, where the first estimated value is used to indicate the probability that the first vector is generated by the expert strategy.

[0078] The discriminator function may also be referred to as a discriminator, which is a commonly used function in reinforcement learning. The present invention does not limit the specific discriminator.

[0079] In the embodiment of the present invention, the first vector and the second vector, the current task queue and the reward function are used as inputs of the value function network. The value function network will output an advantage function, that is, a measure of the advantage of taking the current action in the current state. The embodiment of the present invention uses the temporal difference method to update the value function network. The specific formula can be expressed as follows:

[0080] δ=r+γV(t+1)-V(t) (1)

[0081] Loss = ∑δ 2 (2)

[0082] Formula (1) defines the advantage function δ, where r is the reward value, γ is the discount factor, and V is the value function. These are commonly used parameters and functions in reinforcement learning and will not be described in detail. Formula (2) defines the loss function of the value function network update.

[0083] In the embodiment of the present invention, the policy network of reinforcement learning is used as a generator to generate its own scheduling trajectory (i.e., the first vector), and a discriminator function is used to determine which of the two trajectories is the expert trajectory and which is generated by the policy network. The idea of ​​adversarial training is used to train the trajectory characteristics generated by the policy network to be closer to the trajectory generated by the expert strategy, that is, the discriminator function is used to replace the reward function in reinforcement learning.

[0084] Its specific target loss function can be shown as follows:

[0085] L = -log(P expert )-log(1-P genetal ) (3)

[0086] The second estimated value P expert It represents the probability that the discriminator function considers the true expert trajectory to be the expert trajectory, that is, the probability that the second vector is generated by the expert strategy; the first estimate P general represents the probability that the discriminator function considers the trajectory generated by the strategy to be an expert trajectory, that is, the probability that the first vector is generated by the expert strategy.

[0087] In the embodiment of the present invention, the training goal is to make the discriminator function unable to distinguish the difference between the trajectory generated by the policy network and the real expert, that is, both probability values ​​tend to 0.5. Finally, the score of the policy network trajectory by the discriminator function is used as the reward function r.

[0088] S304: Update the policy network according to the first estimated value, and schedule jobs in the task queue according to the first vector.

[0089] In an embodiment of the present invention, the first estimated value may be input as a reward value into a value function network corresponding to the policy network to obtain an advantage function corresponding to the policy network; and the policy network may be updated according to the advantage function.

[0090] For example, the policy network is updated using the advantage function and the resampling method. The resampling method can scale the sampling results using different strategies to avoid the problem of requiring a large amount of sampling and consuming a lot of time. The specific formula is as follows:

[0091]

[0092]

[0093]

[0094] In formula (4), π and π old Respectively represent the probability of taking action a in state s under the new and old strategies. Formula (5) defines the Loss function used for updating the policy network. Clip means that if α is in the interval (1-∈, 1+∈), it remains unchanged. Otherwise, if it is greater than 1+∈, it takes 1+∈, and if it is less than 1-∈, it takes 1-∈. The value of ∈ is determined according to the actual situation. This method ensures that the change range of the strategy is within a controllable range. Formula (6) is the gradient calculation corresponding to formula (5). min() represents the minimum value operation.

[0095] This method learns the characteristics of the expert strategy generated trajectories on different job sets, so that the reward function can judge the intermediate process of scheduling, thereby making it adaptive to the distribution of different job sets. In addition, it can learn the advantages and disadvantages of the intermediate state of scheduling and improve the convergence speed of the reinforcement learning model training process.

[0096] In the embodiment of the present invention, the following methods can be used to schedule jobs:

[0097] For the jobs in the task queue, the jobs are scheduled into the running queue in descending order of priority according to the first vector until the remaining resources are no longer sufficient to run the job with the highest priority among the remaining jobs in the task queue.

[0098] In the embodiment of the present invention, the jobs in the task queue may be scheduled again at every preset time or when a job in the task queue is completed.

[0099] The above process is described below through a specific embodiment.

[0100] The embodiment of the present invention implements the inverse reinforcement learning scheduling method. In the offline learning process, the embodiment of the present invention uses the collected historical cluster job information for training and learning. In the training process, a certain job is randomly selected as the starting job, and the 255 jobs after it are selected, a total of 256 jobs form a training trajectory, and the trajectory is used as the input of the training.

[0101] After the training starts, the jobs are scheduled every 15 minutes or when a job is completed. During the training process, jobs whose training time is less than the arrival time are considered invisible. A mask function is used, that is, a 256-dimensional vector, where each dimension corresponds to a job, and visible jobs are marked as 1, and invisible jobs are marked as 0.

[0102] Combined with the job duration of each job, the number of CPUs occupied by each job, the number of GPUs occupied by each job, the memory size occupied by each job, and the identifier of each job, all jobs in the task queue can be represented by a (128, 5) matrix. Whenever scheduling is performed, the idle resource information of each node in the cluster is obtained to form an n+m dimensional vector, that is, (the number of idle CPU nodes, the number of idle 2 CPU nodes, ..., the number of idle CPU nodes, the number of idle GPU nodes, the number of idle 2 GPU nodes, ..., the number of idle GPU nodes), where n is the maximum number of CPU cores and m is the maximum number of GPU cards. It is added to the end of each row of the job information matrix to generate a (128, 5+m+n) dimensional matrix S to input into the policy network.

[0103] After passing through the policy network, a 128-dimensional vector A is output, which is the first vector:

[0104] [x1, x2, x3, …, x 128 ]

[0105] where x i Represents the priority of the corresponding job in the task queue, Move jobs into the running queue according to their priority. The higher the priority, the earlier they are moved into the running queue until the remaining resources no longer meet the needs of running the highest priority job in the remaining task queue. At the same time, when waiting for the next scheduling to start, record the state matrix S′.

[0106] Similarly, the expert strategy is used to schedule the matrix S to generate the corresponding second vector A expert For example, if the current expert training focuses on the size of the job, the jobs will be prioritized according to the resources they occupy. The job that occupies less resources has a higher priority. expert Use the discriminator function to evaluate and obtain P general With P expert , use formula (3) to obtain the Loss value, for example, P general =0.5, P expert =0.5, then Loss = 2log2, thereby updating the discriminator function and at the same time general as a reward value.

[0107] The matrix S, the matrix S′, the vector A and the reward value Pgeneral It is passed into the value function network to obtain the output δ of the advantage function. At the same time, after a trajectory is scheduled, the value function network is updated using formulas (7) and (8).

[0108] δ=P general +γV(S′)-V(S) (7)

[0109] Loss = ∑δ 2 (8)

[0110] Use formulas (9), (10), and (11) to update the policy network using the output advantage function δ, thereby completing a round of scheduling and learning:

[0111]

[0112]

[0113]

[0114] It can be seen from the above description that the embodiments of the present invention have the following advantages:

[0115] For a running cluster, a new schedule is performed whenever a job is completed or 15 minutes have passed since the last schedule. The embodiment of the present invention considers both the task queue information and the cluster resource information. During the training process, jobs whose training time is less than the arrival time are considered invisible. A mask function is used, i.e., a 256-dimensional vector, where each dimension corresponds to a job, and visible jobs are marked as 1, and invisible jobs are marked as 0.

[0116] The embodiment of the present invention can represent all jobs in the task queue with a (128, 5) matrix. Whenever scheduling is performed, the idle resource information of each node in the cluster is obtained to form an n+m dimensional vector, that is, [number of idle 1 CPU nodes, number of idle 2 CPU nodes, ..., number of idle nodes for all CPUs, number of idle 1 GPU nodes, number of idle 2 GPU nodes, ..., number of idle nodes for all GPUs], where n is the highest number of CPU cores and m is the highest number of GPU cards. It is added to the end of each row of the job information matrix to generate a (128, 5+m+n) dimensional matrix S to input into the strategy network.

[0117] The embodiment of the present invention can also judge the intermediate process of scheduling, use expert strategy to schedule the matrix, and generate the corresponding action vector A expert , vector A and A expert Use the discriminator function to evaluate and obtain P general With P expert , use the Loss value to update the discriminator function, and at the same time generalAs the reward value, that is, the cluster scheduling optimization target.

[0118] Compared with the existing scheduling implementation for high-performance cluster jobs, the embodiment of the present invention can save a lot of time cost for tuning the cluster priority function. In addition, the embodiment of the present invention can adapt to the distribution of job sets, avoiding the deterioration of scheduling optimization effect due to changes in job set distribution. At the same time, the embodiment of the present invention integrates cluster characteristics and job set characteristics to generate priority functions, so that the embodiment of the present invention can automatically adapt to cluster resource expansion.

[0119] The various embodiments described herein may be independent solutions or may be combined according to internal logic, and all of these solutions fall within the protection scope of this application.

[0120] Based on the same technical concept, an embodiment of the present invention provides a job scheduling device, such as Figure 4 As shown, the device 400 includes: an acquisition unit 401 and a processing unit 402. The division of modules in the embodiments of the present application is schematic, which is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application can be integrated into a processor, or it can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0121] An acquisition unit, used for acquiring job information of a task queue; the task queue includes at least one job;

[0122] A processing unit is used to schedule the task queue according to the job information using a policy network to obtain a first vector, and to schedule the task queue according to the job information using an expert strategy to obtain a second vector; the first vector indicates the priority of each job scheduling in the task queue determined by the policy network, and the second vector indicates the priority of each job scheduling in the task queue determined by the expert strategy; the first vector and the second vector are evaluated by a discriminator function to obtain a first estimated value, and the first estimated value is used to indicate the probability that the first vector is an expert trajectory generated by the expert strategy; the policy network is updated according to the first estimated value, and the jobs in the task queue are scheduled according to the first vector.

[0123] In a possible implementation, the job information includes the following information:

[0124] The submission time of each job in the task queue; the job duration of each job in the task queue; the number of central processing units occupied by each job in the task queue; the number of graphics processing units occupied by each job in the task queue; the memory size occupied by each job in the task queue; and the identification of each job in the task queue.

[0125] In a possible implementation manner, the processing unit is specifically configured to:

[0126] Inputting the first estimated value as a reward value into a value function network corresponding to the policy network to obtain an advantage function corresponding to the policy network;

[0127] The policy network is updated according to the advantage function.

[0128] Based on the same technical concept, an embodiment of the present invention provides a computer device, which may be a terminal or a server. Figure 5 As shown, the device 500 includes at least one processor 501 and a memory 502 connected to the at least one processor. The specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present invention. Figure 5 For example, the processor 501 and the memory 502 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0129] In the embodiment of the present invention, the memory 502 stores instructions that can be executed by at least one processor 501. The at least one processor 501 can execute the above instructions by executing the instructions stored in the memory 502. Figure 5 The steps included in the method shown.

[0130] The processor 501 is the control center of the computer device, and can use various interfaces and lines to connect various parts of the computer device, by running or executing instructions stored in the memory 502 and calling data stored in the memory 502. Optionally, the processor 501 may include one or more processing units, and the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0131] The processor 501 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0132] The memory 502 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 502 is any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 502 in the embodiment of the present invention can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0133] Based on the same inventive concept, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program executable by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned case activity detection method.

[0134] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data transmission device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data transmission device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data transmission device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions may also be loaded onto a computer or other programmable data transmission device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A job scheduling method, characterized in that: include: Get the job information of the task queue; The task queue includes at least one job; The task queue is scheduled according to the job information using a policy network to obtain a first vector, and the task queue is scheduled according to the job information using an expert policy to obtain a second vector; the first vector indicates the priority of each job scheduling in the task queue determined by the policy network, and the second vector indicates the priority of each job scheduling in the task queue determined by the expert policy; Evaluate the first vector and the second vector by a discriminator function to obtain a first estimation value, wherein the first estimation value is used to indicate the probability that the first vector is generated by the expert strategy; Inputting the first estimated value as a reward value into a value function network corresponding to the policy network to obtain an advantage function corresponding to the policy network; The policy network is updated according to the advantage function, and the jobs in the task queue are scheduled according to the first vector.

2. The method according to claim 1, characterized in that The job information includes the following information: The submission time of each job in the task queue; The job duration of each job in the task queue; The number of CPUs occupied by each job in the task queue; the number of GPUs occupied by each job in the task queue; The memory size occupied by each job in the task queue; the identifier of each job in the task queue.

3. The method according to claim 1 or 2, characterized in that The scheduling of the jobs in the task queue according to the first vector includes: For the jobs in the task queue, the jobs are scheduled into the running queue in descending order of priority according to the first vector until the remaining resources are no longer sufficient to run the job with the highest priority among the remaining jobs in the task queue.

4. The method according to claim 1 or 2, characterized in that: The jobs in the task queue are scheduled again at every preset time or when a job in the task queue is completed.

5. A job scheduling device, characterized in that: include: An acquisition unit, used to obtain job information of a task queue; The task queue includes at least one job; A processing unit is used to schedule the task queue according to the job information using a policy network to obtain a first vector, and to schedule the task queue according to the job information using an expert strategy to obtain a second vector; the first vector indicates the priority of each job scheduling in the task queue determined by the policy network, and the second vector indicates the priority of each job scheduling in the task queue determined by the expert strategy; the first vector and the second vector are evaluated by a discriminator function to obtain a first estimated value, and the first estimated value is used to indicate the probability that the first vector is an expert trajectory generated by the expert strategy; the first estimated value is input as a reward value into the value function network corresponding to the policy network to obtain an advantage function corresponding to the policy network; the policy network is updated according to the advantage function, and the jobs in the task queue are scheduled according to the first vector.

6. The device according to claim 5, characterized in that The job information includes the following information: The submission time of each job in the task queue; The job duration of each job in the task queue; The number of CPUs occupied by each job in the task queue; the number of GPUs occupied by each job in the task queue; The memory size occupied by each job in the task queue; the identifier of each job in the task queue.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the job scheduling method according to any one of claims 1 to 4 is implemented.

8. A computer-readable storage medium, characterized in that: The storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images