Job scheduling method, system and equipment and storage medium

Abstract

The invention discloses a job scheduling method, system and device and a storage medium, and belongs to the technical field of oil exploration. Intelligent job scheduling is realized based on the job scheduling system, and the job scheduling system comprises a cloud server, a main proxy server and sub proxy servers. Wherein the cloud server is responsible for carrying out first-level operation disassembly according to operation requirements of a user, and submitting second-level operation obtained after disassembly to each sub-proxy server. And the sub-proxy server can generate a scheduling scheme for the second-level job based on the job scheduling model and submit the scheduling scheme to the main proxy server. And the master proxy server can generate a scheduling scheme of the first-level job by summarizing the scheduling schemes of the second-level jobs and return the scheduling scheme of the first-level job to the user. The job scheduling model is obtained by training by taking optimization of job scheduling time and balancing of equipment load as targets, so that the scheme not only can adapt to personalized job requirements of users, but also can efficiently allocate resources for jobs and balance the equipment load.

Description

Technical Field

[0001] The present application relates to the field of oil exploration, and in particular to an operation scheduling method, system, equipment and storage medium. Background Art

[0002] In the field of oil exploration, with the development of social productivity, users' operational needs are becoming more and more personalized. When operational needs become diverse, it becomes very difficult to reasonably allocate operations based on personalized operational needs and dynamically changing resources. In other words, the complexity and diversity of operational needs and the dynamic changes in resources make it difficult for traditional operation scheduling solutions to simultaneously take into account the requirements of high efficiency in allocating resources for operations and equipment load balancing.

[0003] Based on the above description, it can be seen that in the face of users' personalized job requirements and dynamic changes in resources, how to efficiently allocate resources for jobs, reduce waiting time in queues, and balance equipment loads is a problem that urgently needs to be solved in this field. Summary of the invention

[0004] The embodiment of the present application provides a method, system, device and storage medium for job scheduling. Aiming at the personalized job requirements of users, the solution can not only efficiently allocate resources for jobs, but also balance the equipment load. The technical solution is as follows:

[0005] On the one hand, a job scheduling method is provided, which is applied to a job scheduling system, wherein the job scheduling system includes a cloud server, a plurality of sub-agent servers and a general agent server; the method includes:

[0006] The cloud server decomposes the primary jobs matching the job requirements according to the job requirements submitted by the user to obtain a job set; wherein the job set includes multiple secondary jobs;

[0007] The cloud server sends the job set to the intelligent scheduler of each sub-agent server; wherein the intelligent scheduler includes a resource agent module and a scheduling agent module;

[0008] For the intelligent scheduler of each sub-proxy server, the resource proxy module of the intelligent scheduler prioritizes all secondary jobs currently waiting for resource allocation and releases the secondary job with the highest priority;

[0009] In response to the triggering of a scheduling event, the scheduling agent module of the intelligent scheduler generates a scheduling plan for the released secondary job based on the job scheduling model, and sends the scheduling plan to the general agent server; wherein the job scheduling model is trained based on a training data set with the goal of optimizing job scheduling time and balancing equipment load;

[0010] After receiving the scheduling plans of the multiple secondary jobs included in the primary job, the general agent server generates a scheduling plan for the primary job based on the scheduling plans of the multiple secondary jobs;

[0011] The general agent server returns the scheduling plan of the primary job to the user.

[0012] In a possible implementation, one of the sub-agent servers includes a cluster; the job scheduling model includes a value network and a policy network; and the generating of a scheduling plan for the released secondary job based on the job scheduling model includes:

[0013] The state of the cluster at the scheduling time is input into the strategy network to obtain the probability distribution of the action space; wherein the state space includes multiple states, including: whether it is currently occupied or faulty, the time required to complete the currently assigned job, the difference between the planned start time and the current time, the list of software currently installed on the cluster, the current performance level of the cluster and the time of the current state; the action space includes multiple actions, including: immediate execution, delayed use and dead and unavailable;

[0014] Performing simulation scheduling on the released secondary jobs according to the probability distribution, and obtaining the reward value of the cluster output by the value network;

[0015] Based on the reward value output by the value network, a scheduling plan is generated for the released secondary jobs.

[0016] In a possible implementation, obtaining the reward value of the cluster output by the value network includes:

[0017] determining attenuation parameters;

[0018] Based on the attenuation parameter and the reward function, obtaining a reward value of the fleet;

[0019] Among them, the reward function includes a first part and a second part; the first part and the second part correspond to different weights respectively, and the weights are optimized during model training; the first part is used to represent the reward loss caused by the time required to complete the job and the estimated time required; the second part is used to represent the workload deviation of the cluster; the workload deviation of the cluster is determined based on the load rate of the cluster, the average workload rate of homogeneous clusters and the number of homogeneous clusters.

[0020] In a possible implementation, one of the sub-agent servers includes a cluster; the job scheduling model includes a value network and a policy network;

[0021] The training process of the value network includes:

[0022] Obtaining sample trajectories included in the training data set; wherein the sample trajectories are composed of states, actions, and reward values ​​corresponding to the fleet at different times;

[0023] Under the constraints, the network parameters of the value network are optimized according to the sample trajectories in a batch manner through the loss function of the value network to obtain the trained value network.

[0024] In a possible implementation, the loss function is expressed as follows:

[0025]

[0026] Among them, T ij-rstart represents the actual start time of job j on cluster i, T ij-estart represents the expected start time of job j on cluster i, T ij-rend represents the actual end time of job j on cluster i, T ij-eend represents the expected end time of job j on cluster i, |J| represents the total number of jobs on cluster i, and T ij The time required for job j on cluster i to start and end running.

[0027] In a possible implementation, the generating the scheduling scheme for the primary job based on the scheduling schemes for the plurality of secondary jobs includes:

[0028] Based on the scheduling schemes of the plurality of secondary jobs, generating a scheduling scheme for the primary job with the goal of maximizing the total benefit value of the plurality of secondary jobs;

[0029] Among them, the total benefit value is a comprehensive benefit that integrates the cluster load rate and the job operation efficiency; the scheduling plan of the secondary job includes: the cluster allocated to the secondary job, the start time and end time of the secondary job.

[0030] In a possible implementation, the prioritizing of all secondary jobs currently waiting for resource allocation includes:

[0031] Prioritize all secondary jobs currently waiting for resource allocation based on job submission time, preset job priority, and job scheduled running time;

[0032] Among them, multiple secondary jobs decomposed from the same primary job have the same priority.

[0033] In a possible implementation, the method further includes:

[0034] The general agent server updates the sample tracks generated in the job scheduling process as extended data to the database;

[0035] The general agent server periodically obtains sample trajectories from the database to generate a new training data set; based on the new training data set, the network parameters of the job scheduling model are updated;

[0036] The scheduling agent module of the intelligent scheduler downloads the updated network parameters from the general agent server, and replaces the current network parameters of the job scheduling model with the updated network parameters.

[0037] On the other hand, a job scheduling system is provided, the job scheduling system comprising a cloud server, a plurality of sub-agent servers and a general agent server;

[0038] The cloud server is configured to decompose the primary jobs matching the job requirements submitted by the user to obtain a job set, wherein the job set includes a plurality of secondary jobs;

[0039] The cloud server is further configured to send the job set to the intelligent scheduler of each sub-agent server; wherein the intelligent scheduler includes a resource agent module and a scheduling agent module;

[0040] For the intelligent scheduler of each sub-proxy server, the resource proxy module of the intelligent scheduler prioritizes all secondary jobs currently waiting for resource allocation and releases the secondary job with the highest priority;

[0041] In response to the triggering of a scheduling event, the scheduling agent module of the intelligent scheduler generates a scheduling plan for the released secondary job based on the job scheduling model, and sends the scheduling plan to the general agent server; wherein the job scheduling model is trained based on a training data set with the goal of optimizing job scheduling time and balancing equipment load;

[0042] The general agent server is configured to generate a scheduling scheme for the primary job based on the scheduling schemes of the multiple secondary jobs included in the primary job after receiving the scheduling schemes of the multiple secondary jobs included in the primary job;

[0043] The general proxy server is also configured to return the scheduling plan of the primary job to the user.

[0044] In a possible implementation, one of the sub-agent servers includes a cluster; the job scheduling model includes a value network and a policy network;

[0045] The scheduling agent module is configured as follows:

[0046] The state of the cluster at the scheduling time is input into the strategy network to obtain the probability distribution of the action space; wherein the state space includes multiple states, including: whether it is currently occupied or faulty, the time required to complete the currently assigned job, the difference between the planned start time and the current time, the list of software currently installed on the cluster, the current performance level of the cluster and the time of the current state; the action space includes multiple actions, including: immediate execution, delayed use and dead and unavailable;

[0047] Performing simulation scheduling on the released secondary jobs according to the probability distribution, and obtaining the reward value of the cluster output by the value network;

[0048] Based on the reward value output by the value network, a scheduling plan is generated for the released secondary jobs.

[0049] In a possible implementation, the scheduling agent module is configured as follows:

[0050] determining attenuation parameters;

[0051] Based on the attenuation parameter and the reward function, obtaining a reward value of the fleet;

[0052] Among them, the reward function includes a first part and a second part; the first part and the second part correspond to different weights respectively, and the weights are optimized during model training; the first part is configured to represent the reward loss caused by the time required to complete the job and the estimated time required; the second part is configured to represent the workload deviation of the cluster; the workload deviation of the cluster is determined based on the load rate of the cluster, the average workload rate of homogeneous clusters and the number of homogeneous clusters.

[0053] In a possible implementation, the training process of the value network includes:

[0054] Obtaining sample trajectories included in the training data set; wherein the sample trajectories are composed of states, actions, and reward values ​​corresponding to the fleet at different times;

[0055] Under the constraints, the network parameters of the value network are optimized according to the sample trajectories in a batch manner through the loss function of the value network to obtain the trained value network.

[0056] In a possible implementation, the loss function is expressed as follows:

[0057]

[0058] Among them, Tij-rstart represents the actual start time of job j on cluster i, T ij-estart represents the expected start time of job j on cluster i, T ij-rend represents the actual end time of job j on cluster i, T ij-eend represents the expected end time of job j on cluster i, |J| represents the total number of jobs on cluster i, and T ij The time required for job j on cluster i to start and end running.

[0059] In a possible implementation, the general proxy server is configured as follows:

[0060] Based on the scheduling schemes of the plurality of secondary jobs, generating a scheduling scheme for the primary job with the goal of maximizing the total benefit value of the plurality of secondary jobs;

[0061] Among them, the total benefit value is a comprehensive benefit that integrates the cluster load rate and the job operation efficiency; the scheduling plan of the secondary job includes: the cluster allocated to the secondary job, the start time and end time of the secondary job.

[0062] In a possible implementation, the resource proxy module is configured as follows:

[0063] Prioritize all secondary jobs currently waiting for resource allocation based on job submission time, preset job priority, and job scheduled running time;

[0064] Among them, multiple secondary jobs decomposed from the same primary job have the same priority.

[0065] In a possible implementation, the general proxy server is further configured to update the sample trajectory generated in the job scheduling process as extended data to the database; periodically obtain the sample trajectory from the database to generate a new training data set; and update the network parameters of the job scheduling model according to the new training data set;

[0066] The scheduling agent module is further configured to download updated network parameters from the general agent server and replace the current network parameters of the job scheduling model with the updated network parameters.

[0067] On the other hand, a computer device is provided, the device comprising a processor and a memory, the memory storing at least one program code, the at least one program code being loaded and executed by the processor to implement the above-mentioned job scheduling method.

[0068] On the other hand, a computer-readable storage medium is provided, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to implement the above-mentioned job scheduling method.

[0069] On the other hand, a computer program product or a computer program is provided, which includes a computer program code, and the computer program code is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device executes the above-mentioned job scheduling method.

[0070] The embodiment of the present application implements intelligent job scheduling based on a job scheduling system, which includes a cloud server, a general agent server and a sub-agent server. Among them, the cloud server is responsible for decomposing the job (primary job) according to the job requirements submitted by the user, and submitting the decomposed job (secondary job) to each sub-agent server. The sub-agent server can generate a scheduling plan for the secondary job based on the job scheduling model and submit it to the general agent server. Furthermore, the general agent server can generate a scheduling plan for the primary job by summarizing the scheduling plan of the secondary job and return it to the user. Since the job scheduling model is trained based on the training data set with the goal of optimizing the job scheduling time and balancing the equipment load, the solution can not only adapt to the user's personalized job needs, but also efficiently allocate resources for the job and balance the equipment load, so it is more universal and intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 It is a schematic diagram of an implementation environment involved in a job scheduling method provided in an embodiment of the present application;

[0073] Figure 2 is a flow chart of a job scheduling method provided by an embodiment of the present application;

[0074] Figure 3 It is a flowchart of an intelligent job scheduling provided by an embodiment of the present application;

[0075] Figure 4 is a flow chart of another job scheduling method provided by an embodiment of the present application;

[0076] Figure 5 It is a structural diagram of a job scheduling system provided in an embodiment of the present application;

[0077] Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0079] In this application, the terms "first", "second", etc. are used to distinguish the same or similar items with substantially the same role and function. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0080] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first element can be referred to as a second element, and similarly, a second element can also be referred to as a first element. Both the first element and the second element can be elements, and in some cases, can be separate and different elements.

[0081] Here, at least one means one or more than one, for example, at least one element can be one element, two elements, three elements, or any other integer greater than or equal to one. And multiple means two or more than two, for example, multiple elements can be two elements, three elements, or any other integer greater than or equal to two.

[0082] The "and / or" mentioned in this article indicates that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship.

[0083] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0084] In the related technology, job scheduling algorithms are usually divided into two types, one is a rule-based scheduling algorithm, and the other is a static scheduling method based on a metaheuristic algorithm. Among them, the rule-based scheduling algorithm consists of the shortest processing time, the shortest job priority, etc. Static scheduling methods based on metaheuristic algorithms, such as genetic algorithms, particle swarm optimization algorithms, gray wolf optimization algorithms, etc.; then use search strategies such as crossover, mutation, and particle update to explore the feasible solution space, and after a certain number of iterations, get the optimal solution to the scheduling problem. Although the above method can find the optimal or approximate optimal solution through the search strategy, it lacks adaptability and intelligence.

[0085] In view of personalized job requirements and dynamically changing resources, the embodiment of the present application adopts a reinforcement learning method and proposes an intelligent job scheduling solution based on the Actor-Critic strategy, which can maintain efficient resource allocation for jobs for a long time, reduce waiting time in queues, and balance equipment loads. Among them, reinforcement learning is a popular and very effective artificial intelligence algorithm based on trial and error, which is very suitable for job scheduling optimization problems.

[0086] The following is a detailed introduction to an intelligent job scheduling solution based on the Actor-Critic strategy provided in an embodiment of the present application through the following implementation method.

[0087] Figure 1 It is a schematic diagram of an implementation environment involved in a job scheduling method provided in an embodiment of the present application.

[0088] See also Figure 1 The implementation environment includes: a cloud server 10, a general agent server 11 and a plurality of sub-agent servers 12. The cloud server is also called a cloud platform, the general agent server is also referred to as a general agent, and the sub-agent server is also referred to as a sub-agent.

[0089] It should be noted that Figure 1 Only sub-agent servers AE are listed. In practice, the number of sub-agent servers may be greater or less, and this application does not limit this.

[0090] Combination Figure 2 It can be seen that after the user submits the job requirements to the cloud server 10, the cloud server 10 will disassemble the job according to the job requirements submitted by the user, and send the disassembled job to the sub-agent server 12. In addition, in addition to being responsible for disassembling the job, the cloud server 10 is also responsible for monitoring the equipment workload, production progress and abnormal alarms, etc., which is not limited in this application.

[0091] The proxy server is responsible for controlling various types of cluster resources to complete job scheduling, including the construction of device proxies, negotiation strategies between proxies (this is the responsibility of the general proxy server 11), etc.

[0092] like Figure 1 As shown, the proxy server (including the main proxy server 11 and the sub-proxy server 12) is embedded with an intelligent scheduler (also called an AI scheduler), and the intelligent scheduler further includes a resource proxy module and a scheduling proxy module. Among them, the resource proxy module is mainly responsible for receiving the job order (disassembled job) from the cloud server 10, and prioritizing all jobs currently waiting for resource allocation.

[0093] The scheduling agent module is mainly responsible for generating a job scheduling plan for the jobs released by the resource agent module. In other words, the intelligent scheduler (specifically the scheduling agent module) of the sub-agent server 12 will use the reinforcement learning method to decide the optimal plan, and submit the optimal plan to the general agent server 11 for decision-making, and the general agent server 11 will generate the final job scheduling plan and notify the user of the final job scheduling plan.

[0094] In addition, if Figure 2 As shown, each job scheduling process generates sample trajectories and updates them to the database so that the network parameters of the reinforcement learning network are updated after a certain number of steps. In other words, the intelligent scheduler (specifically the scheduling agent module) periodically requests updated network parameters from the general agent server 11 to update the network parameters of the reinforcement learning network.

[0095] Figure 3 is a flow chart of a job scheduling method provided by an embodiment of the present application. The method is applied to a job scheduling system, wherein the job scheduling system includes a cloud server, multiple sub-agent servers and a general agent server; see Figure 3 , the method comprises the following steps.

[0096] 301. The cloud server decomposes the primary jobs matching the job requirements according to the job requirements submitted by the user to obtain a job set; wherein the job set includes multiple secondary jobs; the cloud server sends the job set to the intelligent scheduler of each sub-agent server; wherein the intelligent scheduler includes a resource agent module and a scheduling agent module.

[0097] In the embodiment of the present application, the job submitted by the user is called a primary job, and after the primary job is decomposed, the decomposed job is called a secondary job. In addition, a sub-agent server includes a cluster; in other words, Figure 1 Each sub-proxy server in the system should be a cluster, so a sub-proxy server can be regarded as a cluster. Among them, a cluster refers to a group composed of different groups.

[0098] Among them, these secondary jobs obtained after disassembly will be sent to all sub-agent servers to allow them to participate in the simulation scheduling. After obtaining the decision plan (i.e., scheduling plan) corresponding to the secondary job, the main agent server will generate the optimal decision plan corresponding to the primary job.

[0099] 302. For the intelligent scheduler of each sub-agent server, the resource agent module of the intelligent scheduler prioritizes all secondary jobs currently waiting for resource allocation and releases the secondary job with the highest priority; in response to the triggering of a scheduling event, the scheduling agent module of the intelligent scheduler generates a scheduling plan for the released secondary job based on the job scheduling model, and sends the generated scheduling plan to the main agent server; wherein the job scheduling model is trained based on a training data set with the goal of optimizing job scheduling time and balancing equipment load.

[0100] Among them, in response to releasing the job, the scheduling event is triggered, that is, the scheduling agent module is triggered to start working. In the embodiment of the present application, the job scheduling model includes a value network and a policy network. The reinforcement learning method based on the Actor-Critic strategy aims to optimize the job scheduling time and balance the equipment load, construct the value network (Critic) and the policy network (Actor), and initialize the network parameters of the two networks, and then use the training data set to continuously optimize the network parameters to obtain the job scheduling model, that is, the reinforcement learning network.

[0101] In addition, all the above-mentioned secondary operations may be secondary operations obtained by decomposing different primary operations, and this application does not limit this. In addition, the priorities of multiple secondary operations decomposed from the same primary operation are consistent.

[0102] In summary, in this step, the resource agent module in the intelligent scheduler sorts the jobs and releases the jobs with the highest priority, thereby triggering the scheduling agent module to start working. The scheduling agent module is responsible for outputting the decision plan corresponding to the secondary job based on the trained job scheduling model.

[0103] 303. After receiving the scheduling plans of multiple secondary jobs included in the primary job, the general agent server generates a scheduling plan for the primary job based on the scheduling plans of multiple secondary jobs included in the primary job, and returns the scheduling plan for the primary job to the user.

[0104] Among them, after the scheduling plan of the first-level job is returned to the user, if the user confirms to use the scheduling plan, the scheduling plan will take effect.

[0105] In addition, resources can be allocated to other primary jobs in the job list by repeatedly executing steps 301-303 until the job list is empty. After a certain number of steps (such as completing resource allocation for a certain number of jobs), the general agent server will update the network parameters of the reinforcement learning network.

[0106] The embodiment of the present application implements intelligent job scheduling based on the reinforcement learning method of the Actor-Critic strategy, which is suitable for complex and diverse job requirements and dynamically changing resources.

[0107] 1. The embodiments of the present application can fully utilize resources in different states to efficiently implement job scheduling, improve the efficiency of allocating resources to jobs, and can adapt to various job requirements.

[0108] 2. The embodiment of the present application fully considers the load of each cluster, and can balance the cluster load while maximizing the efficiency of allocating resources to jobs, thereby ensuring the performance of the cluster while reducing the failure rate of the cluster.

[0109] 3. The embodiment of the present application also supports periodic updating of network parameters of the reinforcement learning network, ensuring the overall efficiency of the scheduling operation plan.

[0110] The above briefly introduces the job scheduling solution provided by the embodiment of the present application. Figure 4 The illustrated embodiment provides a detailed introduction to the job scheduling scheme.

[0111] Figure 4 is a flow chart of another job scheduling method provided by an embodiment of the present application. The method is applied to a job scheduling system, wherein the job scheduling system includes a cloud server, multiple sub-agent servers and a general agent server; see Figure 4 , the method comprises the following steps.

[0112] 401. The general agent server obtains a training data set; based on the training data set, the job scheduling model is trained with the goal of optimizing the job scheduling time and balancing the device load.

[0113] The embodiment of the present application will construct a value network (Critic) and a policy network (Actor). First, the policy network (Actor) is used to continuously interact with the environment to train a good value network (Critic); then, the reward value output by the value network (Critic) is used to update the network parameters of the policy network (Actor) to train a good policy network (Actor).

[0114] The training data set is derived from the historical data records of job scheduling. During model training, the network parameters of the two networks are first initialized, and then the network parameters of the two networks are continuously optimized according to the training data set to generate a job scheduling model (reinforcement learning network) for intelligent job scheduling.

[0115] Exemplarily, the training process of the value network includes, but is not limited to, the following methods:

[0116] Obtain sample trajectories included in the training data set; wherein the sample trajectories are composed of the states s, actions a, and reward values ​​r corresponding to the cluster at different times; and under the constraints, optimize the network parameters of the value network according to the sample trajectories in a batch manner through the loss function of the value network to obtain a trained value network.

[0117] In one possible implementation, the loss function is expressed as follows:

[0118]

[0119] Among them, T ij-rstart represents the actual start time of job j on cluster i, T ij-estart represents the expected start time of job j on cluster i, T ij-rend represents the actual end time of job j on cluster i, T ij-eend represents the expected end time of job j on cluster i, |J| represents the total number of jobs on cluster i, and T ij The time required for job j on cluster i to start and end running.

[0120] In other words, during model training, the embodiment of the present application extracts a series of states, actions, and reward values ​​(s1, a1, r1, ..., s t ,a t ,r t ,…), forming a batch of data, and then optimizing and updating the network parameters of the value network in small batches. In addition, in order to prevent overfitting, constraints are added during the training process to prevent a certain parameter from being too large.

[0121] 402. The cloud server decomposes the primary jobs matching the job requirements according to the job requirements submitted by the user to obtain a job set; wherein the job set includes multiple secondary jobs; the cloud server sends the job set to the intelligent schedulers of each sub-agent server.

[0122] In the embodiment of the present application, when a user submits a personalized job requirement to the cloud platform (for example, a cluster of machines with software A, b clusters with software B, etc. are required), the cloud platform will decompose the job according to the job requirement submitted by the user. For example, job J1 is decomposed into job set {J (1) 1. J (2) 1, ..., J (j) 1}, and then send the job set {J (1) 1. J (2) 1, ..., J (j) 1}.

[0123] Here, j refers to breaking down job J1 into j small jobs, that is, j secondary jobs.

[0124] 403. For the intelligent scheduler of each sub-proxy server, the resource proxy module of the intelligent scheduler prioritizes all secondary jobs currently waiting for resource allocation, releases the secondary job with the highest priority, and the scheduling event is triggered.

[0125] Exemplarily, the resource proxy module prioritizes all secondary jobs currently waiting for resource allocation, including but not limited to the following methods: prioritizing all secondary jobs currently waiting for resource allocation according to job submission time, preset job priority and job scheduled running time.

[0126] The first point to be explained is that the above-mentioned preset job priority can be the priority set by the user, or the default job priority of the system, and this application does not limit this. In addition, the priorities of multiple secondary jobs obtained by decomposing the same primary job are the same. In addition, when sorting jobs, jobs of the same priority are sorted in the order of decomposition. Taking job J1 as an example, that is, keeping {J (1) 1. J (2) 1, ..., J (j) 1} remains in the same order.

[0127] The second point that needs to be explained is that in addition to sending the job set formed by the decomposition of the first-level job to each sub-agent server, the main proxy server can also search for sub-agent servers (such as sub-agent server A and sub-agent server B, etc.) that can complete the job task (all second-level jobs included in the first-level job) among all sub-agent servers, and send the notification of executing the job task to all sub-agent servers that have completed the job task.

[0128] The third point to be explained is that after receiving the notification, these sub-proxy servers will extract and analyze the job tasks contained therein, and obtain the clusters that need to be mobilized (such as the clusters corresponding to sub-proxy servers A and sub-proxy servers B, etc.). In addition, the scheduling agent modules of these sub-proxy servers can also evaluate whether the corresponding cluster is suitable for processing new jobs under the current circumstances based on the queue length of the cluster (whether the time occupied by the already assigned jobs of the same priority or higher priority is too long) and the health status of the cluster (whether a failure occurs). For example, if the queue waiting time of the cluster is too long or a failure occurs, it indicates that the cluster is not currently suitable for undertaking the job task.

[0129] 404. In response to the triggering of the scheduling event, the scheduling agent module of the intelligent scheduler generates a scheduling plan for the released secondary job based on the job scheduling model, and sends the generated scheduling plan to the general agent server.

[0130] In a possible implementation, the scheduling agent module may generate a scheduling plan for the released secondary job based on the job scheduling model in the following manner:

[0131] When a scheduling event is triggered, the state s of the fleet at the scheduling time t is t Input the policy network, and then obtain the probability distribution of the action space; in other words, the decision result is generated based on the state of the fleet at the scheduling time t, that is, the probability distribution of the action space. In the embodiment of the present application, the action space is represented by {a0, a1, a m}, where a0 means it can be executed immediately, a1 means it needs to be used later, and a m Indicates that the system is frozen and unavailable. The probability distribution of the action space is {P0,P1,P m}.

[0132] Next, simulate the scheduling of the released secondary jobs according to the probability distribution to obtain the reward value r of the cluster output by the value network. t , and the reward value r t and the latest status of the fleet t+1 Returns and stores the sample trajectories generated during the job scheduling process.

[0133] The state space includes multiple states. For example, these states include but are not limited to: s1 indicates whether it is currently occupied or faulty, s2 indicates the time required to complete the currently assigned job, s3 indicates the difference between the planned start time and the current time, s4 indicates the software list currently installed in the cluster, s5 indicates the current performance level of the cluster, and s6 indicates the time of the current state.

[0134] In the embodiment of the present application, the job demand is decided by the job scheduling model, and reinforcement learning is performed according to the trained job scheduling model to obtain the reward value of the cluster output by the value network. Then, based on the reward value output by the value network, a scheduling plan is generated for the released secondary job.

[0135] In another possible implementation, the reward value of the cluster output by the value network is obtained, including but not limited to the following methods:

[0136] Determine a decay parameter; obtain a reward value of the fleet based on the decay parameter and the reward function;

[0137] Among them, the reward function includes a first part and a second part; the first part and the second part correspond to different weights respectively, and the weights are optimized during model training; the first part is used to represent the reward loss caused by the time required to complete the job and the estimated time required; the second part is used to represent the workload deviation of the cluster; the workload deviation of the cluster is determined according to the load rate of the cluster, the average workload rate of the homogeneous clusters and the number of homogeneous clusters.

[0138] In the embodiment of the present application, the reward function is divided into two parts, namely, compound reward and penalty. Among them, the compound reward depends on two key factors, namely, the delay time of the job operation and the expected end time. When the cluster cannot complete a job task, it will feedback a larger penalty value to the AI ​​scheduler.

[0139] The goal of reinforcement learning is to maximize the reward value. The expression of the reward function is:

[0140] r t =w i × start +w2×r load

[0141]

[0142]

[0143] Among them, r start Indicates the reward loss caused by the difference between the time required to complete the task and the estimated time required; here M i represents the queued job sequence on cluster i, T it-end It is M i The end time of job i in T it-start is the start time of job i, and the value of i is all natural numbers.

[0144] In addition, load Indicates the workload deviation of the cluster; L i represents the workload rate of cluster i; represents the average workload rate of a homogeneous cluster; M c Indicates the number of homogeneous clusters. Homogeneous clusters are different clusters, but other parameters such as performance are the same.

[0145] It should be noted that when the cluster is unavailable, r t is a relatively large negative penalty value. The reward value is calculated by the following formula:

[0146] R t =r t +γ×r t+1 +...γ T-t × T

[0147] Among them, γ is the attenuation parameter, r T is the reward value at time T, γ T-t is the attenuation parameter at time t, which is γ raised to the power of Tt, where T is the current time plus the total time required for the operation.

[0148] 405. After receiving the scheduling plans of multiple secondary jobs included in the primary job, the general agent server generates a scheduling plan for the primary job based on the scheduling plans of multiple secondary jobs included in the primary job, and returns the scheduling plan for the primary job to the user.

[0149] In the embodiment of the present application, a scheduling scheme for a primary job is generated based on the scheduling schemes for multiple secondary jobs, including but not limited to the following methods:

[0150] Based on the scheduling plans of multiple secondary jobs, a scheduling plan for the primary job is generated with the goal of maximizing the total benefit value of multiple secondary jobs; wherein the total benefit value is a comprehensive benefit that integrates the load rate of the fleet and the efficiency of the job operation; the scheduling plan for the secondary job includes: the fleet allocated to the secondary job, the start time and the end time of the secondary job.

[0151] In other words, after the decision plan corresponding to the secondary job is submitted to the general proxy server, the general proxy server generates the final decision plan according to the following decision rules, that is, the decision plan corresponding to the primary job. The decision rules are as follows:

[0152] (1) The meaning of this formula is to ensure the highest total benefit of global job scheduling.

[0153] (2) The start time and end time of the second-level jobs belonging to the same first-level job are consistent, and then the final decision plan is notified to the user.

[0154] N ij =R t×L i

[0155]

[0156] Here, X i Used to mark a series of clusters, each of which has different software installed and has different performance; T it represents the total time occupied by the queued jobs of the cluster; the efficiency of allocating the same job to different clusters is represented by R t Represents, that is, the reward value mentioned above; N ij It represents the total benefit of allocating the same job to different machine groups.

[0157] For example, the job set {J (1) 1. J (2) 1, ..., J (j) 1} as an example, the steps of job synthesis are as follows:

[0158] 1. For homework J (1) 1, according to max{N (1) i1 Generate decision plan J 1-1 , update the cluster status. Among them, N (1) i1 Represents the benefit value of the first sub-task of job J1.

[0159] 2. For homework J (2) 1. On the premise that the job start time and job end time are consistent, according to max{N (1) i1 +N (2) i1}, Update Scheme J 1-1 For J 1-2 , and update the fleet status at the same time.

[0160] 3. Repeat steps 1 and 2 above until all the tasks J1 are assigned.

[0161] 406. The general proxy server updates the sample trajectory generated during the job scheduling process as extended data to the database.

[0162] The reason why the sample trajectory is updated to the database in this step is to prepare for the subsequent update of the network parameters of the reinforcement learning network.

[0163] It should be noted that the above only introduces job intelligent scheduling by taking a first-level job as an example. For the first-level job with the second highest priority, the above steps can be continued to perform job intelligent scheduling. After a certain number of steps, the general agent server can obtain sample trajectories from the database as training sets and test sets to re-optimize the network parameters of the reinforcement learning network according to the Actor-Critic strategy. In other words, the method provided in the embodiment of the present application also includes:

[0164] The general proxy server periodically obtains sample trajectories from the database to generate a new training data set; based on the new training data set, the network parameters of the job scheduling model are updated; the scheduling agent module of the intelligent scheduler downloads the updated network parameters from the general proxy server and replaces the current network parameters of the job scheduling model with the updated network parameters.

[0165] In summary, the reinforcement learning method based on the Actor-Critic strategy in the embodiment of the present application realizes intelligent job scheduling, which is suitable for complex and diverse job requirements and dynamically changing resources.

[0166] 1. The embodiments of the present application can fully utilize resources in different states to efficiently implement job scheduling, improve the efficiency of allocating resources to jobs, and can adapt to various job requirements.

[0167] 2. The embodiment of the present application fully considers the load of each cluster, and can balance the cluster load while maximizing the efficiency of allocating resources to jobs, thereby ensuring the performance of the cluster while reducing the failure rate of the cluster.

[0168] 3. The embodiment of the present application also supports periodic updating of network parameters of the reinforcement learning network, ensuring the overall efficiency of the scheduling operation plan.

[0169] Figure 5 Schematic diagram of a job scheduling system provided by an embodiment of the present application. Figure 5 The job scheduling system includes a cloud server 501, multiple sub-agent servers 502 and a general agent server 503.

[0170] The cloud server 501 is configured to decompose the primary jobs matching the job requirements submitted by the user to obtain a job set, wherein the job set includes a plurality of secondary jobs;

[0171] The cloud server 501 is further configured to send the job set to the intelligent scheduler of each sub-agent server; wherein the intelligent scheduler includes a resource agent module and a scheduling agent module;

[0172] For the intelligent scheduler of each sub-proxy server 502, the resource proxy module of the intelligent scheduler prioritizes all secondary jobs currently waiting for resource allocation and releases the secondary job with the highest priority; in response to the triggering of a scheduling event, the scheduling proxy module of the intelligent scheduler generates a scheduling plan for the released secondary job based on the job scheduling model, and sends the scheduling plan to the general proxy server; wherein the job scheduling model is trained based on a training data set with the goal of optimizing job scheduling time and balancing equipment load;

[0173] The general agent server 503 is configured to generate a scheduling scheme for the primary job based on the scheduling schemes of the multiple secondary jobs included in the primary job after receiving the scheduling schemes of the multiple secondary jobs included in the primary job;

[0174] The general agent server 503 is also configured to return the scheduling plan of the primary job to the user.

[0175] The embodiment of the present application implements intelligent job scheduling based on a job scheduling system, which includes a cloud server, a general agent server and a sub-agent server. Among them, the cloud server is responsible for decomposing the job (primary job) according to the job requirements submitted by the user, and submitting the decomposed job (secondary job) to each sub-agent server. The sub-agent server can generate a scheduling plan for the secondary job based on the job scheduling model and submit it to the general agent server. Furthermore, the general agent server can generate a scheduling plan for the primary job by summarizing the scheduling plan of the secondary job and return it to the user. Since the job scheduling model is trained based on the training data set with the goal of optimizing the job scheduling time and balancing the equipment load, the solution can not only adapt to the user's personalized job needs, but also efficiently allocate resources for the job and balance the equipment load, so it is more universal and intelligent.

[0176] In a possible implementation, one of the sub-agent servers includes a cluster; the job scheduling model includes a value network and a policy network;

[0177] The scheduling agent module is configured as follows:

[0178] The state of the cluster at the scheduling time is input into the strategy network to obtain the probability distribution of the action space; wherein the state space includes multiple states, including: whether it is currently occupied or faulty, the time required to complete the currently assigned job, the difference between the planned start time and the current time, the list of software currently installed on the cluster, the current performance level of the cluster and the time of the current state; the action space includes multiple actions, including: immediate execution, delayed use and dead and unavailable;

[0179] Performing simulation scheduling on the released secondary jobs according to the probability distribution, and obtaining the reward value of the cluster output by the value network;

[0180] Based on the reward value output by the value network, a scheduling plan is generated for the released secondary jobs.

[0181] In a possible implementation, the scheduling agent module is configured as follows:

[0182] determining attenuation parameters;

[0183] Based on the attenuation parameter and the reward function, obtaining a reward value of the fleet;

[0184] Among them, the reward function includes a first part and a second part; the first part and the second part correspond to different weights respectively, and the weights are optimized during model training; the first part is configured to represent the reward loss caused by the time required to complete the job and the estimated time required; the second part is configured to represent the workload deviation of the cluster; the workload deviation of the cluster is determined based on the load rate of the cluster, the average workload rate of homogeneous clusters and the number of homogeneous clusters.

[0185] In a possible implementation, the training process of the value network includes:

[0186] Obtaining sample trajectories included in the training data set; wherein the sample trajectories are composed of states, actions, and reward values ​​corresponding to the fleet at different times;

[0187] Under the constraints, the network parameters of the value network are optimized according to the sample trajectories in a batch manner through the loss function of the value network to obtain the trained value network.

[0188] In a possible implementation, the loss function is expressed as follows:

[0189]

[0190] Among them, T ij-rstart represents the actual start time of job j on cluster i, T ij-estart represents the expected start time of job j on cluster i, T ij-rend represents the actual end time of job j on cluster i, T ij-eend represents the expected end time of job j on cluster i, |J| represents the total number of jobs on cluster i, and T ij The time required for job j on cluster i to start and end running.

[0191] In a possible implementation, the general proxy server is configured as follows:

[0192] Based on the scheduling schemes of the plurality of secondary jobs, generating a scheduling scheme for the primary job with the goal of maximizing the total benefit value of the plurality of secondary jobs;

[0193] Among them, the total benefit value is a comprehensive benefit that integrates the cluster load rate and the job operation efficiency; the scheduling plan of the secondary job includes: the cluster allocated to the secondary job, the start time and end time of the secondary job.

[0194] In a possible implementation, the resource proxy module is configured as follows:

[0195] Prioritize all secondary jobs currently waiting for resource allocation based on job submission time, preset job priority, and job scheduled running time;

[0196] Among them, multiple secondary jobs decomposed from the same primary job have the same priority.

[0197] In a possible implementation, the general proxy server is further configured to update the sample trajectory generated in the job scheduling process as extended data to the database; periodically obtain the sample trajectory from the database to generate a new training data set; and update the network parameters of the job scheduling model according to the new training data set;

[0198] The scheduling agent module is further configured to download updated network parameters from the general agent server and replace the current network parameters of the job scheduling model with the updated network parameters.

[0199] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0200] It should be noted that: the job scheduling system provided in the above embodiment only uses the division of the above functional components as an example for job scheduling. In actual applications, the above functions can be assigned to different functional components as needed, that is, the internal structure of the system is divided into different functional components to complete all or part of the functions described above. In addition, the job scheduling system provided in the above embodiment and the job scheduling method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0201] Figure 6 It is a structural diagram of a computer device 600 provided in an embodiment of the present application.

[0202] The computer 600 may be a server. The computer device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 601 and one or more memories 602, wherein the memory 602 stores at least one program code, and the at least one program code is loaded and executed by the processor 601 to implement the job scheduling method provided by the above-mentioned various method embodiments. Of course, the computer device 600 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the computer device 600 may also include other components for implementing device functions, which will not be described in detail here.

[0203] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a program code, and the program code can be executed by a processor in a computer device to complete the job scheduling method in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0204] In an exemplary embodiment, a computer program product or a computer program is also provided, which includes a computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device executes the above-mentioned job scheduling method.

[0205] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0206] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A job scheduling method, characterized in that: Applied to a job scheduling system, the job scheduling system includes a cloud server, multiple sub-agent servers and a general agent server; the method includes: The cloud server decomposes the primary jobs matching the job requirements according to the job requirements submitted by the user to obtain a job set; wherein the job set includes multiple secondary jobs; The cloud server sends the job set to the intelligent scheduler of each sub-agent server; wherein the intelligent scheduler includes a resource agent module and a scheduling agent module; For the intelligent scheduler of each sub-proxy server, the resource proxy module of the intelligent scheduler prioritizes all secondary jobs currently waiting for resource allocation and releases the secondary job with the highest priority; In response to the triggering of a scheduling event, the scheduling agent module of the intelligent scheduler generates a scheduling plan for the released secondary job based on the job scheduling model, and sends the scheduling plan to the general agent server; wherein the job scheduling model is trained based on a training data set with the goal of optimizing job scheduling time and balancing equipment load; After receiving the scheduling plans of the multiple secondary jobs included in the primary job, the general agent server generates a scheduling plan for the primary job based on the scheduling plans of the multiple secondary jobs; The general agent server returns the scheduling plan of the primary job to the user.

2. The method according to claim 1, characterized in that One of the sub-agent servers includes a cluster; the job scheduling model includes a value network and a policy network; the generation of a scheduling plan for the released secondary job based on the job scheduling model includes: The state of the cluster at the scheduling time is input into the strategy network to obtain the probability distribution of the action space; wherein the state space includes multiple states, including: whether it is currently occupied or faulty, the time required to complete the currently assigned job, the difference between the planned start time and the current time, the list of software currently installed on the cluster, the current performance level of the cluster and the time of the current state; the action space includes multiple actions, including: immediate execution, delayed use and dead and unavailable; Performing simulation scheduling on the released secondary jobs according to the probability distribution, and obtaining the reward value of the cluster output by the value network; Based on the reward value output by the value network, a scheduling plan is generated for the released secondary jobs.

3. The method according to claim 2, characterized in that The obtaining of the reward value of the cluster output by the value network includes: determining attenuation parameters; Based on the attenuation parameter and the reward function, obtaining a reward value of the fleet; Among them, the reward function includes a first part and a second part; the first part and the second part correspond to different weights respectively, and the weights are optimized during model training; the first part is used to represent the reward loss caused by the time required to complete the job and the estimated time required; the second part is used to represent the workload deviation of the cluster; the workload deviation of the cluster is determined based on the load rate of the cluster, the average workload rate of homogeneous clusters and the number of homogeneous clusters.

4. The method according to claim 1, characterized in that: One of the sub-agent servers includes a cluster; the job scheduling model includes a value network and a strategy network; The training process of the value network includes: Obtaining sample trajectories included in the training data set; wherein the sample trajectories are composed of states, actions, and reward values ​​corresponding to the fleet at different times; Under the constraints, the network parameters of the value network are optimized according to the sample trajectories in a batch manner through the loss function of the value network to obtain the trained value network.

5. The method according to claim 2, characterized in that: The expression of the loss function is as follows: Among them, T ij-rstart represents the actual start time of job j on cluster i, T ij-etart represents the expected start time of job j on cluster i, T ij-rend represents the actual end time of job j on cluster i, T ij-eend represents the expected end time of job j on cluster i, |J| represents the total number of jobs on cluster i, and T ij The time required for job j on cluster i to start and end running.

6. The method according to claim 1, characterized in that The step of generating the scheduling scheme for the primary job based on the scheduling schemes for the plurality of secondary jobs includes: Based on the scheduling schemes of the plurality of secondary jobs, generating a scheduling scheme for the primary job with the goal of maximizing the total benefit value of the plurality of secondary jobs; Among them, the total benefit value is a comprehensive benefit that integrates the cluster load rate and the job operation efficiency; the scheduling plan of the secondary job includes: the cluster allocated to the secondary job, the start time and end time of the secondary job.

7. The method according to claim 1, characterized in that The prioritization of all secondary jobs currently waiting for resource allocation includes: Prioritize all secondary jobs currently waiting for resource allocation based on job submission time, preset job priority, and job scheduled running time; Among them, multiple secondary jobs decomposed from the same primary job have the same priority.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: The general agent server updates the sample tracks generated in the job scheduling process as extended data to the database; The general agent server periodically obtains sample trajectories from the database to generate a new training data set; based on the new training data set, the network parameters of the job scheduling model are updated; The scheduling agent module of the intelligent scheduler downloads the updated network parameters from the general agent server, and replaces the current network parameters of the job scheduling model with the updated network parameters.

9. A job scheduling system, characterized in that: The job scheduling system includes a cloud server, multiple sub-agent servers and a general agent server; The cloud server is configured to decompose the primary jobs matching the job requirements submitted by the user to obtain a job set, wherein the job set includes a plurality of secondary jobs; The cloud server is further configured to send the job set to the intelligent scheduler of each sub-agent server; wherein the intelligent scheduler includes a resource agent module and a scheduling agent module; For the intelligent scheduler of each sub-proxy server, the resource proxy module of the intelligent scheduler prioritizes all secondary jobs currently waiting for resource allocation and releases the secondary job with the highest priority; In response to the triggering of a scheduling event, the scheduling agent module of the intelligent scheduler generates a scheduling plan for the released secondary job based on the job scheduling model, and sends the scheduling plan to the general agent server; wherein the job scheduling model is trained based on a training data set with the goal of optimizing job scheduling time and balancing equipment load; The general agent server is configured to generate a scheduling scheme for the primary job based on the scheduling schemes of the multiple secondary jobs included in the primary job after receiving the scheduling schemes of the multiple secondary jobs included in the primary job; The general proxy server is also configured to return the scheduling plan of the primary job to the user.

10. A computing device, characterized in that The device comprises a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the job scheduling method according to any one of claims 1 to 8.

Images