A yarn cluster workflow scheduling method based on deep reinforcement learning

Abstract

The application discloses a kind of Yarn cluster workflow scheduling methods based on deep reinforcement learning, its characteristics are the method includes: workflow is modeled as directed acyclic graph, it is encoded as workflow state vector using graph neural convolution network, then workflow state vector and Yarn cluster queue resource state vector are jointly input into policy neural network, using proximal policy optimization algorithm to train it etc. steps.Compared with prior art, the application has the advantages of dynamically adjusting scheduling decisions according to the current environment state, improving cluster resource utilization while reducing the overall completion time of the workflow and the delay time of high-priority tasks, the model is simple and easy to train, can significantly improve user experience, and provides technical support for related fields such as Hadoop big data analysis.

Description

Technical Field

[0001] This invention relates to the field of big data and workflow scheduling technology, specifically a deep reinforcement learning-based workflow scheduling method for Yarn clusters used in big data. Background Technology

[0002] Since Google released MapReduce, big data technology has undergone nearly two decades of development and is now widely used in various fields such as business intelligence, healthcare, transportation, the Internet of Things, and artificial intelligence. Hadoop, the open-source distributed computing platform based on the MapReduce programming model, has become the de facto standard in the big data industry. Hadoop 2.0 introduced Yarn, a separate framework for cluster resource management and task scheduling, which supports multiple computing engines such as MapReduce, Spark, and Tez.

[0003] Yarn (Yet Another Resource Negotiator) is a framework that supports multi-user cluster resource management and task scheduling. To prevent interference between different users and to prevent any single user from consuming too many cluster resources, Yarn typically organizes cluster resources into multiple queues, and system administrators can set the maximum available resources for each queue. When a user submits a task, they need to specify the requested resource size (number of CPU cores and memory) and the queue name.

[0004] From Yarn's perspective, tasks submitted to the queue by users are independent, self-executable tasks without any business meaning. However, from the user's perspective, a big data analytics application often consists of several tasks, each handling different business functions, with different priorities, and many tasks having dependencies on each other. A task can only be submitted to the cluster after all its dependent tasks have completed; otherwise, incorrect calculation results may be obtained. Clearly, manually maintaining these dependencies is extremely tedious and complex when there are many tasks and complex dependencies. Therefore, the big data ecosystem introduces a workflow scheduler (also called a workflow management system) to maintain the dependencies between tasks in a workflow and schedule workflows according to these dependencies. Unlike task scheduling on Yarn, the workflow scheduler is only responsible for submitting tasks to the Yarn cluster according to the user-defined task dependencies, and is not responsible for the specific physical resource management and allocation.

[0005] For big data analytics applications, runtime is critical for users. Workflow runtime depends not only on the attributes of the tasks within the workflow (such as computational complexity and the amount of data processed) but also on the available cluster resources. System administrators typically allocate Yarn queues of varying sizes to different users based on their resource needs and priorities. However, setting the maximum available resources for a user's queue is challenging. Because the number, complexity, and dependency structures of tasks within different users' workflows vary, resource requirements often change over time. Setting queue resources too small increases the runtime of a user's tasks, while setting them too large can impact the available resources for other users. In practice, it's common to encounter situations where, for a given period, a queue has reached its maximum resource limit, yet many tasks within that queue are still blocked due to resource shortages, while other queues have ample idle resources. Conversely, the situation may reverse at other times. Yarn is unaware of task dependencies and therefore cannot determine whether directly using these idle resources will affect relevant users. However, if the workflow scheduler can schedule some tasks in a heavily loaded workflow to a queue with available resources without affecting other users, it can further improve resource utilization and reduce workflow runtime.

[0006] The core issue of workflow scheduling is to match appropriate resources to tasks in a workflow to meet certain scheduling objectives, such as minimizing workflow runtime or minimizing resource usage costs, while satisfying task dependencies. Workflow scheduling is currently a hot topic in cloud computing environments. Besides simple traditional workflow algorithms such as priority-based scheduling algorithms and the Earliest Complete Time (HEFT) algorithm in heterogeneous environments, there are also a series of workflow scheduling methods based on heuristic algorithms such as genetic algorithms and ant colony algorithms. Meanwhile, in recent years, due to the good results achieved by machine learning technologies such as deep learning and reinforcement learning in fields such as autonomous driving and robotics, some researchers are now applying these technologies to workflow scheduling. However, different scheduling scenarios have different resource models, scheduling models, and optimization objectives. Workflow scheduling algorithms based on cloud computing environments cannot be directly and effectively applied to big data workflow scheduling on Yarn clusters.

[0007] Currently, commonly used workflow schedulers in the big data ecosystem, such as Azkaban, Ooize, and Airflow, employ very simple scheduling strategies, simply scheduling based on user-defined dependencies without any learning behavior. This often leads to wasted cluster resources and excessively long workflow execution times. Yarn, as one of the most commonly used resource management and task platforms in the big data field, presents a workflow scheduler with a resource model tailored to Yarn, which is of great significance for improving cluster resource utilization and reducing the runtime of big data analytics applications.

[0008] The task scheduler of the existing Yarn cluster is unaware of the dependencies between tasks, and the workflow scheduler does not consider the dynamic changes in cluster queue resources, which leads to problems such as low cluster resource utilization and long workflow execution time. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing a deep reinforcement learning-based workflow scheduler for Yarn clusters. This method employs a deep reinforcement learning-based workflow scheduling approach, taking into account task dependencies and the dynamic status of cluster resources. It dynamically adjusts scheduling decisions based on cluster queue resource availability to improve cluster resource utilization and reduce workflow runtime and latency for high-priority tasks. This effectively solves the problems of Yarn cluster task schedulers failing to perceive task dependencies and workflow schedulers not considering dynamic changes in cluster queue resources, leading to low cluster resource utilization and long workflow runtimes. The method is simple, effective, and can improve cluster resource utilization while reducing overall workflow completion time and latency for high-priority tasks. It provides technical support for Hadoop big data analysis and related fields, and has promising application prospects.

[0010] The specific technical solution to achieve the purpose of this invention is: a Yarn cluster workflow scheduling method based on deep reinforcement learning, which specifically includes the following steps:

[0011] S1, Workflow and Yarn Cluster Resource Modeling

[0012] 1) Workflow: A workflow can be modeled as a directed acyclic graph G = (J, D). Here, J represents the set of tasks in the workflow, J = {j1, j2, ... j}. m}; D represents the set of task dependencies in this workflow, D = {d st |s≠t, and s, t∈{1,...m}}; d st Indicates task j t Depends on task j s , called j sAs a father's task, j t For child tasks; only when the parent task j s After completion, task j t Only then can it start running. Users configure workflows through configuration files, which include: workflow start time, included tasks, fixed attributes of tasks (such as the number of CPU cores, memory size, priority, etc. required to run the task), and dependencies between tasks.

[0013] 2) Yarn Cluster: The system administrator divides the Yarn cluster resources into n queues q1, q2, ..., q n And set the maximum usable resources for each queue as a percentage of the total cluster resources as f1, f2, ..., f n Then, n queues are assigned to users u1, u2, ..., u3. n The queue operates on a first-come, first-served (FIFO) principle, meaning that tasks submitted to the queue first receive resources to run. If the currently used resources in the queue are less than its configured maximum, and the remaining resource quota is sufficient to run a newly submitted task, the task can be executed quickly. Otherwise, the newly submitted task will wait for resources until available resources in the queue meet its running requirements.

[0014] S2, Reinforcement Learning Modeling

[0015] The basic idea of ​​reinforcement learning is that an agent observes the state of its environment, takes corresponding actions to interact with the environment based on the observed state, and continuously optimizes its strategy based on the rewards given by the environment to maximize the expected long-term reward. The three key elements of reinforcement learning are the state space, the action space, and the reward mechanism.

[0016] 1) State Space

[0017] The environmental state S observed by the agent = {S g S c}, S g S represents the workflow status. c Indicates the current status of cluster queue resources; S g ={S g1 S g2 S gn};S gi S represents the state of the i-th workflow; gi It consists of the dependencies between tasks in the i-th workflow and the states of all tasks included in that workflow, i.e., S. gi ={D g S ji , i∈J g}, where task state Sji This includes: fixed attributes and runtime attributes of the task; the fixed attributes include: the amount of resources required by the task and the task priority, etc.; the runtime attributes of the task include: task status, schedulable time, scheduling time, and start time, etc.; the S c ={S q1 S q2 S qn} Among them, S qi ={vcore max vmem max vcore used vmem used vcore max This indicates the maximum number of CPU cores available in the queue; vmem max This indicates the maximum available memory size for the queue; correspondingly, vcore used vmem used These represent the number of CPU cores and memory currently in use for the queue, respectively.

[0018] 2) Action Space

[0019] Based on the S1 workflow and Yarn cluster resource model, the Yarn cluster is divided into n queues {q1,q2,...,q...}. n Let the action space A = {a0, a1, a2, ..., a}. n When the agent performs action a i When, it means that the currently scheduled task will be scheduled into queue q. i Specifically, this invention sets up a virtual queue q0 to indicate that the current task will not be scheduled. That is, when the agent executes action a0, it means that the current task will not be scheduled to any queue. The advantage of designing a virtual queue is that it provides the intelligent scheduler with the ability to delay the scheduling of the current task, thereby potentially making a better scheduling decision from a global perspective.

[0020] 3) Environmental Incentive Mechanism

[0021] Environmental rewards consist of two parts: the single-step reward r during the scheduling process. step And the final reward r at the end of the scheduling final Single-step reward refers to the immediate reward obtained after each scheduling of a task, while final reward refers to the delayed reward obtained after a round of scheduling (all tasks have completed normally or reached the preset maximum time step). If task j is to be scheduled at time step t, the agent executes action a. t The single-step reward obtained afterward is as follows (a):

[0022]

[0023] in, Represents queue q i The available resources are sufficient to run task j.

[0024] Define the delay δ of task j j As shown in equation (b) below:

[0025] δ j =start_time j -avail_time j (b)

[0026] Among them, start_time j avail_time represents the start time of task j. j This represents the schedulable time of a task (the time it takes for all parent tasks to finish running). Let N = |J p＝i | represents the number of tasks with priority p = i among all tasks. The average delay time of all tasks with priority p = i is as shown in equation (c) below:

[0027]

[0028] The final reward r final As shown in equation (d) below:

[0029]

[0030] Among them, w p＝3 This represents the average latency of all tasks with priority 3, with 0.7 as the weighting coefficient. Similarly, w... p＝2 and w p＝1 These represent the average latency of tasks with priorities of 2 and 1, respectively, with weighting coefficients of 0.2 and 0.1. This indicates that the lower the average latency of a task, the greater the reward, and tasks with higher priorities have a larger weighting.

[0031] S3. Workflow scheduling based on the Proximity Policy Optimization (PPO) algorithm

[0032] 1) Neural Network Initialization: The Proximal Policy Optimization (PPO) algorithm is based on an actor-critic architecture. Therefore, three neural networks need to be randomly initialized first, namely the policy network π. θ and the old strategy network And the value network Q. The policy network is used to output the probability of the agent performing each action in a given state, while the value network Q is used to evaluate the quality of the state.

[0033] 2) Iterate through all unstarted workflows in the configuration and check if there is a workflow that needs to be started at the current time t. If the current time t is greater than or equal to the start time of workflow g, then start workflow g. Otherwise, proceed to the next time t+1. After starting workflow g, update the status of the entry task (the task without a parent task) of workflow g to schedulable and add it to the schedulable task queue.

[0034] 3) Check if any task has been completed at the current time t. If task j has been completed, iterate through all subtasks of task j and check the status of the parent task of the subtask. If all parent tasks of the subtask have been completed, update the status of the subtask to be schedulable and add it to the schedulable task queue.

[0035] 4) Randomly select a task from the schedulable task queue and update its isActive value to 1, indicating that this task is the task that needs to be scheduled at the current time t.

[0036] 5) Graph Convolutional Neural Network (GCN) is used to embed the workflow state into a fixed-size vector, which is then combined with the Yarn cluster state to form the current observable state s of the intelligent scheduler. t .

[0037] 6) Set state s t Input to policy network π θ In the process, the policy network outputs the probability of all actions, and then samples are performed based on these probabilities to obtain action a. t Schedule the current task to queue q. i (i = a t Simultaneously, the isActive value of the current task is restored to 0, and then the current reward r is calculated according to the environment reward mechanism in step S2 reinforcement learning modeling. t If the scheduling has not ended at time t, then r t =r step Otherwise r t =r final and {s t a t r t Stored in the memory pool.

[0038] 7) Proceed to the next time step t+1 and repeat steps 2) to 6) until the preset deadline t is reached. max Alternatively, the maximum number of steps per round, T, or all tasks in all workflows have finished running.

[0039] 8) Retrieve data {s0, a0, r0, s1, a1, r1...s} from the memory pool. T a T r T}, will state s tInput into the value network Q, and use the value network to estimate ν(s) t Then, the action advantage A is calculated using equations (e) and (f) as follows. t :

[0040] μ t =r t +γν(s t+1 )-ν(s t (e);

[0041] A t =μ t +(γλ)μ t+1 +...+(γλ) T-t+1 μ T-1 (f).

[0042] Wherein, γ and λ are hyperparameters; in this invention, γ is taken as 0.99, λ as 0.95, and r t ν(s) represents the reward obtained at time t. t ) and ν(s t+1 ) represent the state values ​​at time t and time t+1, respectively.

[0043] 9) Retrieve data {s0, a0, r0, s1, a1, r1...s} from the memory pool. T a T r T}, will state s t The inputs are respectively fed into the policy network π. θ and policy network We obtain the results in state s respectively t The action probability distribution π θ (a t |s t ),and The importance weight r is calculated according to the following formula (g). t (θ), and then L is calculated using the following formula (h). clip (θ) is used as the loss function for backpropagation to update the policy network π. θ :

[0044]

[0045]

[0046] Where ε is a hyperparameter, and in this invention, ε = 0.2, clip(r t (θ), 1-ε, 1+ε) represent when r t When (θ) is greater than 1+ε, it takes the value 1+ε, and when r tWhen (θ) is less than 1-ε, it takes the value 1-ε. The min function represents taking the smaller of the two. It indicates a desire for the expected value.

[0047] 10) Repeat steps 8-9) several times, then overwrite the old policy network parameters with the new policy network parameters, i.e., let

[0048] 11) Calculate the discount return G according to the following formula (i). t Then use the discount reward G t The predicted value ν(s) of the value network Q t The difference is calculated, and the mean squared error (MSE) is used as the loss function. Backpropagation is then performed to update the value network parameters.

[0049] G t =r t+1 +γr t+2 +γ 2 r t+3 +...+γ T-t r T+1 +γ T-t+1 ν(s t+1 (i)

[0050] 12) Clear the memory pool and repeat steps (2)-(11) several times until the environmental reward obtained by the intelligent scheduler converges to the optimal or suboptimal value.

[0051] Compared with existing technologies, this invention achieves higher cluster resource utilization, reduces the runtime of individual user workflows, and lowers the average latency of tasks. Furthermore, it can dynamically adjust scheduling decisions based on changes in cluster resources; for example, when resources are scarce, it prioritizes high-priority tasks to ensure their runtime. The model is simple, easy to train, and significantly improves user experience, providing technical support for the field of big data workflow scheduling. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the deep reinforcement learning model of the present invention;

[0053] Figure 2 This is a comparison chart of the workflow execution time of this invention with other scheduling algorithms. Detailed Implementation

[0054] The present invention will be further described and illustrated in detail below with specific embodiments:

[0055] Example 1

[0056] See Figure 1 Perform workflow scheduling on the Yarn cluster using the following steps:

[0057] S1, Workflow and Yarn Cluster Resource Modeling

[0058] 1) Workflow: A workflow can be modeled as a directed acyclic graph G = (J, D), where J represents the set of tasks in the workflow, J = {j1, j2, ... j}. m}; D represents the set of task dependencies in this workflow, D = {d st |s≠t,and s,t∈{1,...m}};d st Indicates task j t Depends on task j s , called j s As a father's task, j t For a child task, only when the parent task j s After completion, task j t Only then can it start running. Users configure workflows through configuration files, which include: workflow start time, included tasks, fixed attributes of tasks (such as the number of CPU cores, memory size, priority, etc. required to run the task), and dependencies between tasks.

[0059] 2) Yarn Cluster: The system administrator divides the Yarn cluster resources into n queues q1, q2, ..., q n And set the maximum usable resources for each queue as a percentage of the total cluster resources as f1, f2, ..., f n Then, n queues are assigned to users u1, u2, ..., u3. n The queue operates on a first-come, first-served (FIFO) principle, meaning that tasks submitted to the queue first receive resources to run. If the currently used resources in the queue are less than its configured maximum, and the remaining resource quota is sufficient to run a newly submitted task, the task can be executed quickly. Otherwise, the newly submitted task will wait for resources until available resources in the queue meet its running requirements.

[0060] S2, Reinforcement Learning Modeling

[0061] The basic idea of ​​reinforcement learning is that an agent observes the state of its environment, takes corresponding actions to interact with the environment based on the observed state, and continuously optimizes its strategy based on the rewards given by the environment to maximize the expected long-term reward. The three key elements of reinforcement learning are the state space, the action space, and the reward mechanism.

[0062] 1) State Space

[0063] The environmental state S observed by the agent = {S g S c}, where Sg Indicates the workflow status; S c Indicates the current status of cluster queue resources; S g ={S g1 S g2 S gn}. S gi S represents the state of the i-th workflow; gi It consists of the dependencies between tasks in the i-th workflow and the states of all tasks included in that workflow, i.e., S. gi ={D g S ji , i∈J g Task status includes: fixed attributes and runtime attributes; fixed attributes include: the amount of resources required by the task and the task priority, while runtime attributes include task status, schedulable time, scheduling time, start time, etc. c ={S q1 S q2 S qn}, S qi ={vcore max vmem max vcore used vmem used vcore max This indicates the maximum number of CPU cores available in the queue, vmem. max This indicates the maximum available memory size for the queue. Correspondingly, vcore used vmem used These represent the number of CPU cores and memory currently in use for the queue, respectively.

[0064] 2) Action Space

[0065] Based on the S1 workflow and Yarn cluster resource model, the Yarn cluster is divided into n queues {q1, q2, ..., q...}. n Let the action space A = {a0, a1, a2, ..., a}. n When the agent performs action a i When, it means that the currently scheduled task will be scheduled into queue q. i Specifically, this invention sets up a virtual queue q0 to indicate that the current task will not be scheduled. That is, when the agent executes action a0, it means that the current task will not be scheduled to any queue. The advantage of designing a virtual queue is that it provides the intelligent scheduler with the ability to delay the scheduling of the current task, thereby potentially making a better scheduling decision from a global perspective.

[0066] 3) Environmental Incentive Mechanism

[0067] Environmental rewards consist of two parts: the single-step reward r during the scheduling process. step And the final reward r at the end of the scheduling final Single-step reward refers to the immediate reward obtained after each scheduling of a task, while final reward refers to the delayed reward obtained after a round of scheduling (all tasks have completed normally or reached the preset maximum time step). If task j is to be scheduled at time step t, the agent executes action a. t The subsequent single-step reward is calculated using the following formula (a):

[0068]

[0069] in, Represents queue q i The available resources are sufficient to run task j.

[0070] Define the delay δ of task j j Calculated by the following formula (b):

[0071] δ j =start_time j -avail_time j (b)

[0072] Among them, start_time j avail_time represents the start time of task j. j This represents the schedulable time of a task (the time it takes for all parent tasks to finish running). Let N = |J p＝i | represents the number of tasks with priority p = i among all tasks. The average delay time of all tasks with priority p = i is calculated by the following formula (c):

[0073]

[0074] The final reward r final Calculated from the following (d):

[0075]

[0076] Among them, w p＝3 This represents the average latency of all tasks with priority 3, with 0.7 as the weighting coefficient. Similarly, w... p＝2 and w p＝1 These represent the average latency of tasks with priorities of 2 and 1, respectively, with weighting coefficients of 0.2 and 0.1. This indicates that the lower the average latency of a task, the greater the reward, and tasks with higher priorities have a larger weighting.

[0077] S3. Workflow scheduling based on the Proximity Policy Optimization (PPO) algorithm

[0078] 1) Neural Network Initialization. The Proximal Policy Optimization (PPO) algorithm is based on an actor-critic architecture. Therefore, three neural networks need to be randomly initialized first, namely the policy network π. θ and the old strategy network And the value network Q. The policy network is used to output the probability of the agent performing each action in a given state, while the value network is used to evaluate the quality of the state.

[0079] 2) Iterate through all unstarted workflows in the configuration and check if there is a workflow that needs to be started at the current time t. If the current time t >= the start time of workflow g, then start workflow g. Otherwise, proceed to the next time t+1. After starting workflow g, update the status of the entry task of workflow g (the task without a parent task) to schedulable and add it to the schedulable task queue.

[0080] 3) Check if any task has been completed at the current time t. If task j has been completed, iterate through all subtasks of task j and check the status of the parent task of the subtask. If all parent tasks of the subtask have been completed, update the status of the subtask to be schedulable and add it to the schedulable task queue.

[0081] 4) Randomly select a task from the schedulable task queue and update its isActive value to 1, indicating that this task is the task that needs to be scheduled at the current time t.

[0082] 5) Graph Convolutional Neural Network (GCN) is used to embed the workflow states into a fixed-size vector. This vector is then combined with the Yarn cluster state to form the current observable state s of the intelligent scheduler. t .

[0083] 6) Set state s t Input to policy network π θ In the process, the policy network outputs the probability of all actions, and then samples are performed based on these probabilities to obtain action a. t Schedule the current task to queue q. i (i = a t Simultaneously, the isActive value of the current task is restored to 0, and then the current reward r is calculated according to the environment reward mechanism in the reinforcement learning modeling of step S2. t If the scheduling has not ended at time t, then r t =r step Otherwise r t =r final and {s t a t rt Stored in the memory pool.

[0084] 7) Proceed to the next time step t+1 and repeat steps 2) to 6) until the preset deadline t is reached. max Alternatively, the maximum number of steps per round, T, or all tasks in all workflows have finished running.

[0085] 8) Retrieve data {s0, a0, r0, s1, a1, r1...s} from the memory pool. T a T r T}, will state s t Input into the value network Q, and use the value network Q to estimate ν(s) t Then, the action advantage A is calculated using equations (e) and (f) below. t :

[0086] μ t =r t +γν(s t+1 )-ν(s t (e);

[0087] A t =μ t +(γλ)μ t+1 +...+(γλ) T-t+1 μ T-1 (f).

[0088] Wherein, γ and λ are hyperparameters; in this invention, γ is taken as 0.99, λ as 0.95, and r t ν(s) represents the reward obtained at time t. t ) and ν(s t+1 ) represent the state values ​​at time t and time t+1, respectively.

[0089] 9) Retrieve data {s0, a0, r0, s1, a1, r1...s} from the memory pool. T a Y r T}, will state s t The inputs are respectively fed into the policy network π. θ and policy network We obtain the results in state s respectively t The action probability distribution π θ (a t |s t ),and The importance weight r is calculated according to the following formula (g). t (θ), and then calculate L using the following formula (h). clip (θ) is used as the loss function for backpropagation to update the policy network π. θ:

[0090]

[0091]

[0092] Where ε is a hyperparameter, and in this invention, ε = 0.2, clip(r t (θ), 1-ε, 1+ε) represent when r t When (θ) is greater than 1+ε, it takes the value 1+ε, and when r t When (θ) is less than 1-ε, it takes the value 1-ε. The min function represents taking the smaller of the two. It indicates a desire for the expected value.

[0093] 10) Repeat steps 8-9 several times, then overwrite the old policy network parameters with the new policy network parameters, i.e., let

[0094] 11) Calculate the discount return G according to the following formula (i). t Then use the discount reward G t The predicted value ν(s) of the value network Q t The difference is calculated, and the mean squared error (MSE) is used as the loss function. Backpropagation is then performed to update the value network parameters.

[0095] G t =r t+1 +γr t+2 +γ 2 r t+3 +...+γ T-t r T+1 +γ T-t+1 ν(s t+1 (i).

[0096] 12) Clear the memory pool and repeat steps 2 to 11) several times until the environmental reward obtained by the intelligent scheduler converges to the optimal or suboptimal value.

[0097] See Figure 2 PPO corresponds to the Yarn cluster workflow scheduling method based on deep reinforcement learning in this invention. Simulation results show that the scheduling algorithm in this invention can achieve lower workflow running time compared with static scheduling algorithm, random algorithm, priority-based scheduling algorithm (PBS) and earliest completion time algorithm (HEFT).

[0098] The above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A workflow scheduling method for Yarn clusters based on deep reinforcement learning, characterized in that, The method specifically includes the following steps: 1) Collect workflow information and cluster resource information based on the user's workflow configuration and Yarn cluster queue configuration, and construct a workflow scheduling model based on the collected information; 2) Based on the workflow scheduling model and Yarn cluster resource model, reinforcement learning modeling is performed, including the design of state space, action space and reward mechanism; 3) The scheduler is trained using the proximal policy optimization algorithm in deep reinforcement learning algorithms; The reinforcement learning model in step 2) includes: a) State space The environmental state S observed by the agent = { }; in, Indicates the status of the workflow. ={ }; Indicates the current status of cluster queue resources. ={ }; = { }; This indicates the maximum number of CPU cores available in the queue; This indicates the maximum available memory size for the queue; These represent the number of CPU cores and memory currently in use by the queue, respectively. This represents the state of the i-th workflow. ; The task state includes fixed attributes and runtime attributes; the fixed attributes of the task include: the amount of resources required by the task and the task priority; the runtime attributes of the task include: task state, schedulable time, scheduling time and start time. b) Action space The Yarn cluster is divided into n queues. Let the action space A = { When the agent performs an action When, it indicates that the currently scheduled task will be scheduled into the queue. ; c) Environmental incentive mechanism Environmental rewards include: single-step rewards during the scheduling process. And the final reward upon completion of the scheduling. The single-step reward is the immediate reward obtained after each scheduling of a task. If task j is to be scheduled at time step t, the agent executes the action. The subsequent single-step reward is calculated using the following formula (a): （a）； in Queue The available resources are sufficient to run task j; otherwise, define the delay time for task j. Calculated by the following formula (b): （b）； in, This represents the start time of task j. The schedulable time for the task; The final reward is a delayed reward obtained after all tasks have been completed normally or have reached the preset maximum time step. , represents the number of tasks with priority p=i out of all tasks, then the average latency of all tasks with priority p=i is... Calculated by the following formula (c): （c）； The final reward Calculated by the following formula (d): （d）； in, This represents the average latency of all tasks with priority 3, with a weighting factor of 0.

7. Similarly... and These represent the average latency of tasks with priorities of 2 and 1, respectively, with weighting coefficients of 0.2 and 0.

1.

2. The Yarn cluster workflow scheduling method based on deep reinforcement learning according to claim 1, characterized in that, The construction of the workflow scheduling model in step 1) includes: a) Model a workflow as a directed acyclic graph G=(J,D); Where J represents the set of tasks in this workflow, J={ }; D represents the set of task dependencies in this workflow, D={ }; Subtasks Depends on parent task Only when the parent task After completion, subtask Only then can it start running; b) Users configure workflows through configuration files. The configuration includes: workflow start time, included tasks, fixed attributes of tasks, and dependencies between tasks. The fixed attributes of a task include: the number of CPU cores, memory size, and priority required to run the task. c) The workflow scheduler schedules tasks from the user-configured workflow to the Yarn cluster, which is divided into n queues { }, and assign n queues to users. The maximum percentage of total cluster resources that each queue can use is [percentage missing]. .

3. The Yarn cluster workflow scheduling method based on deep reinforcement learning according to claim 1, characterized in that, Step 3) of training the scheduler using the near-end policy optimization algorithm specifically includes: a) For the current policy network Old policy network The current policy network is randomly initialized with three neural networks: the value network Q, the value network Q, and the network itself. and the old strategy network The value network Q is used to evaluate the quality of a state and outputs the probability of each action performed by the agent in a given state. b) Traverse all unstarted workflows g in the configuration, check if there is a workflow g that needs to be started at the current time t. If the current time t is greater than or equal to the start time of workflow g, start workflow g, update the status of the entry task of workflow g to schedulable, and add it to the schedulable task queue; otherwise, proceed to the next time t+1. c) Check if task j has completed at the current time t. If task j has completed, then iterate through all subtasks of task j. Check subtasks Parent task The state of the subtask All parent tasks Once all tasks are completed, then the subtask will be... The status is updated to schedulable, and it is added to the schedulable task queue; d) Randomly select a task j from the schedulable task queue and update its isActive value to 1, indicating that this task j is the task j that needs to be scheduled at the current time t; e) Use a graph convolutional neural network (GCN) to embed the state of workflow g into a fixed-size vector, and then combine it with the Yarn cluster state to form the state observable by the current intelligent scheduler. ; f) Change the state Input Policy Network According to the policy network Sampling is performed on the probabilities of all actions to obtain the actions. Schedule the current task j to the queue. At the same time, the isActive value of the current task is restored to 0, and then the current reward is calculated according to the environment reward mechanism. If the scheduling has not ended at time t, then ,otherwise Finally, Stored in the memory pool; g) Proceed to the next time step t+1 and repeat steps b) to f) until the preset deadline is reached. The maximum number of steps T per round or all tasks j in all workflows g have finished running; h) Retrieve data from the memory pool , will state Inputting into the value network Q yields the state. Then, the action advantage is calculated using the following formula (e). : （e）； in, For hyperparameters; Let (f) represent: （f）； in, This represents the reward obtained at time t; and These represent the state values ​​at time t and time t+1, respectively. i) Retrieve data from the memory pool , will state Input into the policy network respectively and policy network , respectively obtained and Calculate importance weights and with As a loss function, backpropagation updates the policy network. The importance weight Calculated by the following formula (g): （g）； The From the following formula (h) ; ] （h）； in, For hyperparameters; Indicates when Greater than When, the value is ,when When, the value is The min function takes the smaller of the two values. This indicates a demand for expectation; j) Repeat steps h) to i) several times, then use a policy network. Parameters cover the old policy network Parameters, i.e., let ; k) Using discount rewards With the predicted value of the value network Q Using the mean squared error (MSE) as the loss function, backpropagation is performed to update the Q parameters of the value network, and the discounted reward is... Calculated by the following formula (i): （i）； l) Clear the memory pool and repeat steps b to k) several times until the environmental reward obtained by the intelligent scheduler converges to the optimal or suboptimal value.

4. The Yarn cluster workflow scheduling method based on deep reinforcement learning according to claim 3, characterized in that, In step b), the Yarn cluster is divided into n queues. A virtual queue was set up in} This is used to indicate that the current task j will not be scheduled, i.e., { When the agent performs an action When the time is specified, it means that the current task j will not be scheduled to any queue.

Images