Cluster job planning method and system based on dual time window mechanism

Abstract

The invention discloses a cluster job planning method and system based on a dual time window mechanism. The method comprises the steps that jobs are divided into a multi-level structure with an execution sequence, and a directed acyclic graph is constructed to form a task topological structure; constructing a dual time window mechanism; designing a multi-objective optimization function; modeling cluster job scheduling into a multi-agent Markov decision process, setting a hierarchical action space, and designing a reward function reflecting scheduling balance; the method comprises the following steps: constructing a hybrid neural network architecture, extracting a time sequence correlation feature of a task, encoding a heterogeneous feature between the task and a server by using a heterogeneous graph neural network, and generating an intelligent matching score between the task and the server; a multi-agent near-end strategy optimization algorithm is applied, and agent collaborative optimization is achieved through a collaborative mechanism updated by a centralized value function and a distributed strategy. According to the method, fine modeling and state dynamic perception of complex operation topology are realized, and scientificity and controllability of scheduling decision are improved.

Description

Technical Field

[0001] This invention relates to the fields of edge computing and resource scheduling, specifically to a cluster job planning method and system based on a dual time window mechanism. Background Technology

[0002] With the rapid development of cloud computing, edge computing, and big data technologies, cluster job scheduling and resource allocation have become core issues in improving the overall performance of data centers and distributed computing platforms. Currently, cluster scheduling systems such as Kubernetes, YARN, and Mesos are widely used for container management and task orchestration. These systems typically employ static rules or heuristic algorithms (such as first-come, first-served, resource scoring, affinity scheduling, etc.) to meet the basic scheduling requirements of regular container tasks.

[0003] However, in practical applications, especially in scenarios such as batch processing and AI training pipelines, jobs often involve complex multi-level dependencies, and the execution time and required computing units of each subtask can be determined before scheduling. Taking Kubernetes as an example, its scheduler is mainly based on node resource filtering and optimization scoring, lacking the ability to deeply model and optimize inter-task dependencies, job topology, and task execution order. It is difficult to fully utilize the deterministic information of tasks and to balance multiple objectives such as resource utilization, scheduling timeliness, and system load balancing.

[0004] In recent years, reinforcement learning, especially multi-agent reinforcement learning and proximal policy optimization, has been introduced into the field of cluster scheduling. These methods can adaptively optimize scheduling strategies through continuous interaction with the environment, improving the scheduling quality of the system under dynamic loads and complex constraints. However, existing research mostly focuses on theoretical simulations or simplified models, and has not yet developed efficient scheduling solutions for real-world scenarios with multi-level job dependencies and fixed task execution times and resource requirements.

[0005] Furthermore, low resource utilization is a common problem in current cluster environments, mainly due to the imbalance between task and resource allocation, coarse scheduling granularity, and a lack of refined management of task diversity and resource heterogeneity. Therefore, there is an urgent need for a scheduling method that integrates task dependency analysis, dynamic resource state modeling, and intelligent strategy optimization to fully utilize the determinism of task execution time and resource requirements, thereby improving cluster resource utilization, task completion timeliness, and system throughput. Summary of the Invention

[0006] The purpose of this invention is to provide a cluster job planning method and system based on a dual time window mechanism. In response to the problems of low resource utilization, slow scheduling response, and uneven system load in existing scheduling systems under scenarios where task execution time and required computing units are fixed, an efficient, intelligent, and adaptive scheduling optimization scheme is proposed.

[0007] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: a cluster job planning method based on a dual time window mechanism, comprising the following steps:

[0008] Step 1: Based on the dependencies between subtasks in the job, divide the entire job into a multi-level structure with execution order and construct a directed acyclic graph to form the task topology.

[0009] Step 2: Construct a dual time window mechanism. The static time window is used to statistically analyze the server's operating status within a fixed time period, while the dynamic time window is adaptively adjusted based on real-time feedback information from the candidate task queue.

[0010] Step 3: Design a multi-objective optimization function that integrates factors such as task processing timeliness, resource utilization, and system load balancing, and introduce time constraints and resource constraints into the scheduling decision to optimize the overall scheduling performance.

[0011] Step 4: Model the cluster job scheduling as a multi-agent Markov decision process, set up a hierarchical action space for job and server running status, task allocation and execution mode, and design a reward function that reflects the scheduling balance.

[0012] Step 5: Construct a hybrid neural network architecture, use graph neural networks to extract the temporal correlation features of the task, use heterogeneous graph neural networks to encode the heterogeneous characteristics between the task and the server, and combine a Transformer-based pointer network to generate an intelligent matching score between the task and the server.

[0013] Step 6: Apply the multi-agent proximal policy optimization algorithm to achieve agent-based collaborative optimization through a collaborative mechanism of centralized value function and distributed policy update.

[0014] Furthermore, in step S1, decomposing the job into a multi-level structure means constructing jobs with sub-task dependencies into a structured model with a clear hierarchical division according to the dependencies between tasks. Each task in each level can only be executed after all its preceding level tasks are completed. Specifically, this includes the following steps:

[0015] Step S11. Initialize the task status, set all tasks in the job to the waiting state, construct a set of tasks to be processed, and set the initial level number;

[0016] Step S12. In the current level, identify all tasks whose preceding dependent tasks have been completed, and designate them as "executable" tasks of this level; if there are no "executable" tasks in the current level but the set of tasks to be processed is not empty, then it is determined that there is a circular dependency and the processing is terminated.

[0017] Step S13. Assign the executable tasks of the current level to the task set of the corresponding level, update their status to "completed", and remove them from the pending set; increment the level number by one, and repeat the task filtering process of step S12 until all tasks are processed.

[0018] Step S14. Output the constructed multi-level task structure to support subsequent scheduling optimization and resource matching.

[0019] The method for constructing a directed acyclic graph to form a task topology graph is as follows: divide the job into a multi-level structure according to steps S11 to S14, and then connect the tasks in each level to the dependent tasks of the level above it through directed edges, thereby forming a directed acyclic graph structure with clear dependencies.

[0020] In step S1, the subtask running status is divided as follows: waiting task, which refers to a task whose predecessor dependent tasks have not been completed and which does not yet meet the scheduling conditions; task to be executed, which refers to a task whose dependent tasks have been completed but have not yet been scheduled for execution; candidate task, which refers to a task to be executed in the scheduling process of the scheduler; task in execution, which refers to a task that is currently being actually executed on the server; and completed task, which refers to a task that has completed all computation and resource release processes.

[0021] The dual time window used in step S2 includes a static time window and a dynamic time window;

[0022] Among them, the static time window refers to a time window with a fixed length t. sw The time periods divided into segments on the time axis are used to statistically analyze the server's operational status within those fixed time periods.

[0023] A dynamic time window refers to a window that extends from the current time t to the future time t+t. dw A continuous time interval is used to assist in evaluating the scheduling feasibility of candidate tasks; the length t of the dynamic time window. dw The value of n is determined by the number of candidate tasks. w Maximum time window length t wmax With minimum time window length t wmin It is jointly decided and can be dynamically adjusted according to the system status. The calculation formula is as follows:

[0024] t dw =f(n) w ,t wmax,t wmin ).

[0025] Within this dynamic time window, all pending tasks that have completed their prerequisite dependent tasks will be included in the candidate task set for the scheduler to make scheduling decisions.

[0026] The dynamic adjustment of the dynamic time window includes the following steps:

[0027] Step S21: Calculate the cardinality of the current candidate task set and compare it with the preset number of candidate tasks n. w Conduct a comparative evaluation to determine whether the current candidate tasks are inadequate;

[0028] Step S22: When the cardinality of the candidate task set is less than n w In this process, the upper bound of the dynamic time window is gradually expanded to continuously absorb potential candidate tasks until the maximum time span t is reached. wmax Or the number of candidate tasks reaches the preset value n w until;

[0029] Step S23: When the cardinality of the candidate task set reaches or exceeds n w At this time, the length of the dynamic time window can be appropriately shortened, but the minimum time span t must be maintained. wmin This is to avoid invalid resource scanning or scheduling delays;

[0030] Step S24: Adaptively adjust the length t of the dynamic time window based on the above steps. w The candidate task pool is updated in real time to ensure that the dynamic time window overlaps with at most two consecutive static time windows.

[0031] In step S3, considering time constraints and resource constraints, a multi-objective optimization model integrating factors such as task processing timeliness, resource utilization, and system load balancing is constructed as follows:

[0032]

[0033] in Let be the completion time of task i. Given its expected latest completion time, This is an indicator function; its value is 1 if the condition is true, and 0 otherwise. m,k (t) represents the usage of the k-th type of resource on server m at time t. U represents the total capacity of this resource on server m; m This represents the average resource utilization rate of the m-th computing node. The value represents the average utilization of all nodes; T represents the total time span, and M represents the total number of nodes; the start time of task i is denoted as . The earliest start time of task i is denoted as The set of all its preceding tasks is Pre(i); if task i is assigned to server m for execution, then a im =1, otherwise a im =0.

[0034] Furthermore, in step S4, the cluster job scheduling is modeled as a multi-agent Markov decision process, and the hierarchical action space for job and server running states, task allocation and execution methods is set, including:

[0035] The running status of each job can be set, including: whether it is completed, the remaining execution time, the ideal execution time period, whether there are predecessor tasks, whether there are successor tasks, the amount of various resources required, and the execution time required.

[0036] Setting the server's running status includes: window status information, which describes the discrete distribution characteristics of the server's running status within the first intersecting static time window and the subsequent static time window when dynamic and static time windows overlap.

[0037] The discrete distribution characteristics include the minimum and maximum values ​​of availability, utilization, and runtime for each type of resource group on the server. The calculation method is as follows: Assume the server has K types of computing resources, and the total capacity of each type of resource is denoted as R. k The computing units of this type of resource are sorted according to availability, and the sorting results are divided into G sub-resource groups to form dynamic resource groups; the utilization rate and availability rate of resources are calculated in each sub-resource group; the statistical process is carried out independently in the above two static time windows.

[0038] In step S4, the task allocation and execution method adopts hierarchical action space modeling, specifically including the following:

[0039] Within each scheduling time step, each server acts as an agent, selecting a task from its candidate task set and deciding on the execution method based on its current running state and the state of the selected task.

[0040] For the selected task, if the j-th execution method is selected, the server selects available computing resources from the resources in the j-th sub-resource group after sorting. Under the premise of satisfying the time constraint, the server determines the start execution time of the task and performs corresponding resource allocation scheduling according to the greedy strategy of earliest completion time first.

[0041] Preferably, in step S4, a reward function reflecting scheduling balance is designed, specifically including the following: scheduling balance includes task processing timeliness, resource utilization, and load balancing. Each server (agent) has a reward function, as shown in the following formula:

[0042]

[0043] Where α var α is the coefficient for balanced reward. util η1 is the utilization reward coefficient, η2 is the contribution of static time window 1 to the balance reward, and η2 is the utilization reward contribution of static time window 1. β is the task timeout penalty coefficient, and t... i It is the time required for server i to execute the selected task at time step t, where t is the time required. i,start It is the time when the task execution starts, t i,ddl This is the latest expected deadline for the task to complete. i,t (s i,t ,a i,t ) is the reward that server i receives at time step t, s i,t This refers to the local state of server i at time step t, including the running state of server i and the states of all its selectable tasks; a i,t It refers to the task selection made by server i at time step t and the method of executing the task. and These are the static time window 1 balance reward and utilization reward. and These are the static time window 2 balance bonus and utilization bonus, calculated as follows:

[0044]

[0045] in It represents the overall resource utilization of server i within a static time window of 1. It is the utilization rate of the k-th computing resource of server i within a static time window of 1. It is the average overall resource utilization of M servers within a static time window of 1. It represents the overall resource utilization of server i within a static time window of 2. It is the utilization rate of the k-th computing resource of server i within a static time window of 2. It is the average overall resource utilization of M servers in static time window 2;

[0046] The total reward is the arithmetic average of the rewards from M servers:

[0047]

[0048] Furthermore, in step S5, the Bayesian posterior mechanism generates a conflict-free candidate task set for each server, specifically as follows:

[0049] Step S51: Calculate the matching degree value between each server and each candidate task using a heterogeneous graph neural network and a pointer network based on the Transformer structure;

[0050] Step S52: Remove the matching degree values ​​from the computation graph of the neural network, and calculate the affinity score of each task to each server using the Bayesian posterior formula. The affinity calculation formula is as follows:

[0051]

[0052] Where Pr(i|j) represents the matching degree value of server j in selecting task i, and Pr(j|i) represents affinity, which means the probability that task i belongs to server j. It represents the prior probability that server j will accept the task, and indicates the resource idle rate in the current static time window 1 and static time window 2.

[0053] Step S53: Based on the matching degree between the server and the task, select candidate tasks for each server using probability sampling. If multiple servers sample the same task, it is considered a conflict. Then, conflict mediation is performed according to the affinity score described in step S52. Only the server-task pairing with the highest affinity is retained, and the task in the pairing is added to the candidate task set of the corresponding server.

[0054] Step S54: Repeat the sampling and conflict resolution process until the current candidate task queue is divided into multiple non-overlapping candidate task sets, each corresponding to a server, thereby generating a non-overlapping set of server candidate tasks for subsequent task scheduling decisions.

[0055] This invention also provides a cluster job planning system based on a dual time window mechanism, used to implement the above-mentioned cluster job planning method. The system includes:

[0056] The first module is used to divide the entire job into a multi-level structure with execution order based on the dependencies between subtasks in the job and to construct a directed acyclic graph to form a task topology.

[0057] The second module is used to construct a dual time window mechanism. The static time window is used to statistically analyze the server's running status within a fixed time period, while the dynamic time window is adaptively adjusted based on the real-time feedback information from the candidate task queue.

[0058] The third module is used to design a multi-objective optimization function that integrates factors such as task processing timeliness, resource utilization and system load balancing, and introduces time constraints and resource constraints into scheduling decisions to optimize overall scheduling performance.

[0059] The fourth module is used to model cluster job scheduling as a multi-agent Markov decision process, setting up a hierarchical action space for job and server running status, task allocation and execution methods, and designing a reward function that reflects scheduling balance.

[0060] The fifth module is used to build a hybrid neural network architecture. It uses graph neural networks to extract the temporal correlation features of tasks, uses heterogeneous graph neural networks to encode the heterogeneous characteristics between tasks and servers, and combines a Transformer-based pointer network to generate intelligent matching scores between tasks and servers.

[0061] The sixth module applies a multi-agent proximal policy optimization algorithm, which achieves agent-based collaborative optimization through a collaborative mechanism of centralized value function and distributed policy update.

[0062] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention divides the job into a multi-level structure and clarifies the dependency relationship between sub-tasks based on the directed acyclic graph. Combined with the fact that the task execution time and required computing units are fixed, the present invention realizes fine modeling and dynamic state perception of complex job topology, which improves the scientificity and controllability of scheduling decision; (2) The present invention adopts a dual time window mechanism. The static time window is used to periodically count the server resource status, and the dynamic time window adaptively adjusts the scheduling range according to the real-time feedback of the candidate task queue. It can dynamically balance the scheduling timeliness and resource utilization, adapt to the changes in cluster load, significantly improve the overall resource utilization efficiency of the system, and can be applied to offline scheduling and online scheduling; (3) The present invention models the cluster job scheduling as a multi-agent Markov decision process. Combined with the hybrid neural network structure and the multi-agent near-end policy optimization algorithm, it realizes distributed collaborative scheduling between servers. It can take into account the task completion timeliness, resource utilization and load balancing, improve the system throughput, and is particularly suitable for scenarios where the task execution time and resource requirements are known before scheduling. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the method steps of the present invention.

[0064] Figure 2 This is a schematic diagram illustrating the task hierarchy dependency of the present invention.

[0065] Figure 3 This is a schematic diagram of the dual time window mechanism of the present invention.

[0066] Figure 4This is a schematic diagram of the network model structure of the present invention.

[0067] Figure 5 This is a schematic diagram illustrating the specific process of the method of the present invention.

[0068] Figure 6 This is a diagram illustrating the algorithm solution and training process architecture in this invention.

[0069] Figure 7 The diagram shows the actual effect provided by the present invention. Detailed Implementation

[0070] To provide a clearer understanding of the features and advantages of the technical solution of the present invention, the composition and implementation of the specific solution are described below in conjunction with the accompanying drawings.

[0071] Reference Figure 1 , Figure 2 , Figure 3 , Figure 5 This invention proposes a cluster job planning method based on a dual time window mechanism, comprising the following steps:

[0072] Step S1: Based on the dependencies between subtasks in the job, divide the entire job into a multi-level structure with a strict execution order. Construct a directed acyclic graph (DAG) based on this structure to form a task topology graph that clarifies the causal relationships between tasks. During task execution, based on the progress of subtasks and the completion status of their dependencies, classify tasks into five states: waiting task, task to be executed, candidate task, task in execution, and completed task.

[0073] Step S2: Construct a dual time window mechanism. The static time window is used to statistically analyze the server's operating status within a fixed time period. The dynamic time window is adaptively adjusted based on the real-time feedback information from the candidate task queue to achieve a dynamic balance between the timeliness of task scheduling and the efficiency of resource utilization.

[0074] Step S3: Considering time and resource constraints, construct a multi-objective optimization model that integrates factors such as task processing timeliness, resource utilization, and system load balancing to optimize overall scheduling performance.

[0075] Step S4: Model the cluster job scheduling as a multi-agent Markov decision process, set up a hierarchical action space for job and server running status, task allocation and execution mode, and design a reward function that reflects the scheduling balance.

[0076] Step S5: Construct a hybrid neural network architecture, including: using graph neural networks to extract temporal correlation features of tasks; utilizing heterogeneous graph neural networks to encode the heterogeneous characteristics between tasks and servers; and combining a Transformer-based pointer network to generate intelligent matching scores between tasks and servers. A conflict-free candidate task set is generated for each server using a Bayesian posterior mechanism, and sampling is performed to determine the final task allocation scheme. An independent execution policy network is used to formulate specific plans for servers to execute the allocated tasks.

[0077] Step S6: Apply the multi-agent proximal policy optimization algorithm to achieve agent collaborative optimization through the collaborative mechanism of centralized value function and distributed policy update. This enables dynamic perception of task status, adaptive scheduling under resource constraints, and distributed collaborative decision-making, thereby improving cluster resource utilization and task completion timeliness, balancing server load, and increasing system throughput.

[0078] According to an embodiment of the present invention, in step S1, decomposing the job into a multi-level structure means constructing jobs with sub-task dependencies into a structured model with a clear hierarchical division according to the dependencies between tasks. Each task in each level can only be executed after all its preceding level tasks have been completed. Specifically, this includes the following steps:

[0079] Step S11. Initialize the task status, set all tasks in the job to the waiting state, construct a set of tasks to be processed, and set the initial level number;

[0080] Step S12. In the current level, identify all tasks whose preceding dependent tasks have been completed, and designate them as "executable" tasks of this level; if there are no "executable" tasks in the current level but the set of tasks to be processed is not empty, then it is determined that there is a circular dependency and the processing is terminated.

[0081] Step S13: Assign the executable tasks of the current level to the task set of the corresponding level, update their status to "completed", and remove them from the pending set; increment the level number by one, and repeat the task filtering process in step b until all tasks are processed.

[0082] Step S14: Output the constructed multi-level task structure to support subsequent scheduling optimization and resource matching.

[0083] The method for constructing a directed acyclic graph to form a task topology graph is as follows: divide the job into a multi-level structure according to steps S11 to S14, and then connect the tasks in each level to the dependent tasks of the level above it through directed edges, thereby forming a directed acyclic graph structure with clear dependencies.

[0084] According to an embodiment of the present invention, in step S1, the division of the running state of the subtask includes: waiting task, which refers to a task whose predecessor dependent tasks have not been completed and which does not yet meet the scheduling conditions; task to be executed, which refers to a task whose dependent tasks have been completed but have not yet been scheduled for execution; candidate task, which refers to a task to be executed in the scheduling process of the scheduler; task in execution, which refers to a task that is currently being actually executed on the server; and completed task, which refers to a task that has completed all computation and resource release processes.

[0085] According to an embodiment of the present invention, the dual time window used in step S2 includes a static time window and a dynamic time window;

[0086] Among them, the static time window refers to a time window with a fixed length t. sw The time periods divided into segments on the time axis are used to statistically analyze the server's operational status within those fixed time periods.

[0087] A dynamic time window refers to a window that extends from the current time t to the future time t+t. dw A continuous time interval is used to assist in evaluating the scheduling feasibility of candidate tasks; the length t of the dynamic time window. dw The value of n is determined by the number of candidate tasks. w Maximum time window length t wmax With minimum time window length t wmin It is jointly decided and can be dynamically adjusted according to the system status. The calculation formula is as follows:

[0088] t dw =f(n) w ,t wmax ,t wmin )

[0089] Within this dynamic time window, all pending tasks that have met the dependency conditions will be included in the candidate task set for the scheduler to make scheduling decisions.

[0090] The process of determining the dynamic time window includes the following steps:

[0091] Step S21: Calculate the cardinality of the current candidate task set and compare it with the preset number of candidate tasks n. w Conduct a comparative evaluation to determine whether the current candidate tasks are inadequate;

[0092] Step S22: When the cardinality of the candidate task set is less than n w In this process, the upper bound of the dynamic time window is gradually expanded to continuously absorb potential candidate tasks until the maximum time span t is reached. wmax Or the number of candidate tasks reaches the preset value n w until;

[0093] Step S23: When the cardinality of the candidate task set reaches or exceeds n w At this time, the length of the dynamic time window can be appropriately shortened, but the minimum time span t must be maintained. wmin This is to avoid invalid resource scanning or scheduling delays;

[0094] Step S24: Adaptively adjust the length t of the dynamic time window based on the above steps. w The candidate task pool is updated in real time to ensure that the dynamic time window intersects with at most two consecutive static time windows. The current number of candidate tasks is counted and compared with the target number of candidate tasks, n. w Compare them to determine whether the current candidate task is in a state of insufficiency;

[0095] According to an embodiment of the present invention, in step S3, considering time constraints and resource constraints, a multi-objective optimization model integrating factors such as task processing timeliness, resource utilization, and system load balancing is constructed as follows:

[0096]

[0097] in Let d be the completion time of task i. i For its expected completion time, This is an indicator function; its value is 1 if the condition is true, and 0 otherwise. k (t) represents the usage of the k-th type of resource at time t, R k r represents the total capacity of the resource. i k Indicates the amount of this type of resource used by task i; U m This represents the average resource utilization rate of the m-th computing node. Let T represent the average utilization of all nodes; T represents the total time span; and M represents the total number of nodes. The start time of task i is denoted as . The set of all its preceding tasks is Pre(i). If task i is assigned to server m for execution, then a im =1, otherwise a im =0.

[0098] According to an embodiment of the present invention, in step S4, the cluster job scheduling is modeled as a multi-agent Markov decision process, and the hierarchical action space for job and server running states, task allocation and execution methods includes:

[0099] The running status of each job can be set, including: whether it is completed, the remaining execution time, the ideal execution time period, whether there are predecessor tasks, whether there are successor tasks, the amount of various resources required, and the execution time required.

[0100] Setting the server's running status includes: window status information, which describes the discrete distribution characteristics of the server's running status within the first intersecting static time window and the subsequent static time window when dynamic and static time windows overlap.

[0101] The discrete distribution characteristics include the minimum and maximum values ​​of availability, utilization, and runtime for each type of resource group on the server. The calculation method is as follows: Assume the server has K types of computing resources, and the total capacity of each type of resource is denoted as R. k The computing units of this type of resource are sorted according to availability, and the sorting results are divided into G sub-resource groups to form dynamic resource groups; the utilization rate and availability rate of resources are calculated in each sub-resource group; the statistical process is carried out independently in the above two static time windows.

[0102] According to an embodiment of the present invention, in step S4, the task allocation and execution method adopts hierarchical action space modeling, specifically including the following:

[0103] Within each scheduling time step, each server acts as an agent, selecting a task from its candidate task set and deciding on the execution method based on its current running state and the state of the selected task.

[0104] For the selected task, if the j-th execution method is selected, the server selects available computing resources from the resources in the j-th sub-resource group after sorting. Under the premise of satisfying the time constraint, the server determines the start execution time of the task and performs corresponding resource allocation scheduling according to the greedy strategy of earliest completion time first.

[0105] According to an embodiment of the present invention, in step S4, a reward function reflecting scheduling balance is designed, specifically including the following:

[0106] Scheduling balance includes task processing timeliness, resource utilization, and load balancing. Each server (agent) has a reward function, as shown in the following formula:

[0107]

[0108] Where α var α is the coefficient for balanced reward. util η1 is the utilization reward coefficient, η2 is the contribution of static time window 1 to the balance reward, and η2 is the utilization reward contribution of static time window 1. β is the task timeout penalty coefficient, and t... i It is the time required for server i to execute the selected task at time step t, where t is the time required. i,start It is the time when the task execution starts, t i,ddlThis is the latest expected deadline for the task to complete. i,t (s i,t ,a i,t ) is the reward that server i receives at time step t, s i,t This refers to the local state of server i at time step t, including the running state of server i and the states of all its selectable tasks; a i,t It refers to the task selection made by server i at time step t and the method of executing the task. and These are the static time window 1 balance reward and utilization reward. and These are the static time window 2 balance bonus and utilization bonus, calculated as follows:

[0109]

[0110] in It represents the overall resource utilization of server i within a static time window of 1. It is the utilization rate of the k-th computing resource of server i within a static time window of 1. It is the average overall resource utilization of M servers within a static time window of 1. It represents the overall resource utilization of server i within a static time window of 2. It is the utilization rate of the k-th computing resource of server i within a static time window of 2. It is the average overall resource utilization of M servers in static time window 2;

[0111] The total reward is the arithmetic average of the rewards from M servers:

[0112]

[0113] According to an embodiment of the present invention, in step S5, the Bayesian posterior mechanism generates a conflict-free candidate task set for each server, specifically as follows:

[0114] Step S51: Calculate the matching degree value between each server and each candidate task using a heterogeneous graph neural network and a pointer network based on the Transformer structure;

[0115] Step S52: Remove the matching degree values ​​from the computation graph of the neural network, and calculate the affinity score of each task to each server using the Bayesian posterior formula. The affinity calculation formula is as follows:

[0116]

[0117] Where Pr(i|j) represents the matching degree value of server j in selecting task i, and Pr(j|i) represents affinity, which means the probability that task i belongs to server j. It represents the prior probability that server j will accept the task, and indicates the resource idle rate in the current static time window 1 and static time window 2.

[0118] Step S53: Based on the matching degree between the server and the task, select candidate tasks for each server using probability sampling. If multiple servers sample the same task, it is considered a conflict. Then, conflict mediation is performed according to the affinity score described in step S52. Only the server-task pairing with the highest affinity is retained, and the task in the pairing is added to the candidate task set of the corresponding server.

[0119] Step S54: Repeat the sampling and conflict resolution process until the current candidate task queue is divided into multiple non-overlapping candidate task sets, each corresponding to a server, thereby generating a non-overlapping set of server candidate tasks for subsequent task scheduling decisions.

[0120] Figure 6 This is a schematic diagram of the training process of the present invention. The steps of the algorithm training process are explained as follows:

[0121] Step S1: Environment Interaction and Data Acquisition. In each round of training, the agents interact with the cluster environment based on the current policy network (Actor). Each agent selects an action based on its own observations (including task status, server status, etc.), which constitutes the decision on task allocation and execution within the current scheduling cycle. The environment provides feedback on new states and rewards based on the agents' actions. Multiple agents collect interaction trajectories in parallel, forming a complete training sample set.

[0122] Step S2: Experience storage and batch sampling: All collected interaction data (including state, action, reward, next state, etc.) are stored in the experience pool or batch cache for subsequent centralized training and batch optimization.

[0123] Step S3: Advantage estimation and reward calculation: Using a global centralized Critic network, the collected trajectory data is evaluated for value, and methods such as generalized advantage estimation are used to calculate the advantage value and target value of each action, providing reference signals for updating the policy network.

[0124] Step S4: Update the policy network (Actor) and value network (Critic)

[0125] The policy network is updated based on the MAPPO algorithm. It uses the advantage value obtained from sampling to optimize the Actor network parameters through gradient ascent, so that the policy network can output better task allocation and execution method selection.

[0126] Value Network Update: By minimizing the mean square error between predicted value and actual return, the Critic network parameters are optimized to improve the accuracy of state value assessment.

[0127] Step S5: Distributed collaboration and parameter synchronization. Multiple agents can interact with the environment and update policies in parallel. During training, global information sharing and value evaluation are achieved through centralized Critic. The network parameters of each agent are synchronized periodically to achieve distributed collaborative optimization.

[0128] Step S6: Repeat the above steps to continuously optimize the agent's policy network and value network. By monitoring indicators such as global reward, resource utilization, and task completion timeliness, determine whether the training has converged or achieved the expected performance, and finally output the optimal scheduling strategy.

[0129] Preferably, the training objective of the MAPPO algorithm network is defined as:

[0130] 1. Policy (Actor) Loss Function:

[0131]

[0132] Where θ is the parameter of the Actor network; B is the batch size, i.e. the number of samples used in each training session; and M is the number of servers. It is the policy probability ratio, where sample i contains empirical tuples from M servers sampled at time step t in the environment. It is the state code of server m in sample i at time step t, which includes the running state code of server m and the state codes of all its selectable tasks; It represents the action of server m in sample i, indicating the task selection and execution method of server m; It is based on observations when interacting with the environment. Get the action to be executed The probability, During neural network training, the Actor network encodes the state of sample i. Get the action to be executed The probability of [the value]. clip is the truncation function, calculated using the following formula:

[0133]

[0134] Where ∈ is the policy cutoff coefficient, which is usually taken as ∈ = 0.2.

[0135] Let be the advantage function of the i-th sample (belonging to agent m, corresponding to time step t), and the recursive formula for its calculation is:

[0136]

[0137] γ is the discount factor, and λ is the attenuation factor for the advantage estimate. This is the TD error; σ is the entropy coefficient, used to encourage exploration. It is the entropy of the strategy.

[0138] 2. Critic Loss Function:

[0139]

[0140] Where φ is the parameter of the Critic network; It is the current value network pair of states The predicted value; It is the old value network (i.e., the Critic network obtained from the previous round or several iterations) for the state. The predicted value, R i,t It is the return for sample i, calculated using the following formula:

[0141] Preferably, the overall value function estimation uses the mean of the local value function estimated by each server, i.e. Where s t Let V(s) be the global state at time step t. t ) represents the actual state value.

[0142] Table 1: Hybrid Neural Network Structure

[0143]

[0144] Preferably, the hybrid decision network is divided into the following modules: In the topology association feature extraction module, the capability value dimension of the nodes is first embedded into a 16-dimensional vector using an embedding layer, and then two layers of graph attention (GAT) are used to further extract task features. In the server-task heterogeneous feature extraction module, different types of nodes are embedded into a 64-dimensional vector using an embedding layer, and then two layers of heterogeneous GAT + residual connections are used for deep feature extraction. The task selection pointer network module uses a Transformer Encoder and additive attention mechanism to output the probability of each node being selected. The task execution mode selection module outputs the execution mode of the task selected by the server through an MLP layer. The structure of each module is shown in Table 1, and the data transfer between each network module is referenced. Figure 4 .

[0145] To verify the performance of the proposed method, the following experiment was conducted. Six computing servers (each with one type of computing resource) were designed, with 150 jobs to be processed, categorized into three types of sequential jobs. The computation time and computing units required for the subtasks of each job are shown in Table 2. To improve robustness, some ongoing tasks were randomly added to the servers before assignment began. During training, 50 jobs were randomly generated each time for training.

[0146] Table 2 Task Phase Parameters

[0147]

[0148]

[0149] Figure 7 The diagram shows the results of the experiment, illustrating the utilization distribution of the six servers. It can be seen that the method of this invention can distribute the task intensity more evenly across each server, resulting in a low variance in utilization, with each server achieving a resource utilization rate of over 70%.

[0150] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the cluster job scheduling method based on a dual time window mechanism and multi-agent proximal policy optimization as described above.

[0151] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the cluster job scheduling method based on a dual time window mechanism and multi-agent proximal strategy optimization as described above.

[0152] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus (systems), computer devices, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] This invention is described with reference to a flowchart of a method according to embodiments of the invention. It should be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A device for a function specified in one or more processes.

[0154] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 The function specified in one or more processes.

[0155] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 Steps of a specified function in one or more processes.

Claims

1. A cluster job planning method based on a dual time window mechanism, characterized in that, Includes the following steps: Step 1: Based on the dependencies between subtasks in the job, divide the entire job into a multi-level structure with execution order and construct a directed acyclic graph to form the task topology. Step 2: Construct a dual time window mechanism. The static time window is used to statistically analyze the server's operating status within a fixed time period, while the dynamic time window is adaptively adjusted based on real-time feedback information from the candidate task queue. Step 3: Design a multi-objective optimization function that integrates factors such as task processing timeliness, resource utilization, and system load balancing, and introduce time constraints and resource constraints into the scheduling decision to optimize the overall scheduling performance. Step 4: Model the cluster job scheduling as a multi-agent Markov decision process, set up a hierarchical action space for job and server running status, task allocation and execution mode, and design a reward function that reflects the scheduling balance. Step 5: Construct a hybrid neural network architecture, use graph neural networks to extract the temporal correlation features of the task, use heterogeneous graph neural networks to encode the heterogeneous characteristics between the task and the server, and combine a Transformer-based pointer network to generate an intelligent matching score between the task and the server. Step 6: Apply the multi-agent proximal policy optimization algorithm to achieve agent-based collaborative optimization through a collaborative mechanism of centralized value function and distributed policy update.

2. The cluster job planning method based on a dual time window mechanism according to claim 1, characterized in that, In step 1, dividing the entire job into a multi-level structure with an execution order means constructing subtasks with dependencies into a structured model with a clear hierarchical division according to the dependencies between tasks. Each task in each level can only be executed after all its preceding level tasks have been completed. Specifically, this includes the following steps: Step S11: Initialize the task status, set all tasks in the job to the waiting state, construct the set of tasks to be processed, and set the initial level number; Step S12: In the current level, identify all tasks whose preceding dependent tasks have been completed, and designate them as executable tasks for this level; if there are no executable tasks in the current level but the set of tasks to be processed is not empty, then it is determined that there is a circular dependency and the processing is terminated. Step S13: Assign the executable tasks of the current level to the task set of the corresponding level, update their status to completed, and remove them from the pending set; increment the level number by one, and repeat the task filtering process of step 12 until all tasks are processed. Step S14: Output the constructed multi-level task structure to support subsequent scheduling optimization and resource matching; The method for constructing a directed acyclic graph to form a task topology graph is as follows: according to the aforementioned multi-level structure, the tasks in each level are connected to the dependent tasks of the level above through directed edges, thereby forming a directed acyclic graph structure with clear dependencies.

3. The cluster job planning method based on a dual time window mechanism according to claim 1, characterized in that, In step 1, the subtask running status is divided as follows: waiting task, which refers to a task whose predecessor dependent task has not been completed and does not meet the scheduling conditions; task to be executed, which refers to a task whose dependent task has been completed but has not yet been scheduled for execution; candidate task, which refers to a task to be executed in the scheduling process of the scheduler; task in execution, which refers to a task that is currently being actually executed on the server; and completed task, which refers to a task that has completed all computing and resource release processes.

4. The cluster job planning method based on a dual time window mechanism according to claim 1, characterized in that, In step 2, the dual time window used includes a static time window and a dynamic time window; Among them, the static time window refers to a time window with a fixed length t. sw The time periods divided into segments on the time axis are used to statistically analyze the server's operational status within those fixed time periods. A dynamic time window refers to a window that extends from the current time t to the future time t+t. dw A continuous time interval is used to assist in evaluating the scheduling feasibility of candidate tasks; the length t of the dynamic time window. dw The value of n is determined by the number of candidate tasks. w Maximum time window length t wmax With minimum time window length t wmin The decision is made jointly and dynamically adjusted according to the system status. The calculation formula is as follows: t dw =f(n w ,t wmax ,t wmin ). Within the dynamic time window, all pending tasks that have completed their prerequisites will be included in the candidate task set for the scheduler to make scheduling decisions. The dynamic adjustment of the dynamic time window includes the following steps: Step S21: Calculate the cardinality of the current candidate task set and compare it with the preset number of candidate tasks n. w Conduct a comparative evaluation to determine whether the current candidate tasks are inadequate; Step S22: When the cardinality of the candidate task set is less than n w In this process, the upper bound of the dynamic time window is gradually expanded to continuously absorb potential candidate tasks until the maximum time span t is reached. wmax Or the number of candidate tasks reaches the preset value n w until; Step S23: When the cardinality of the candidate task set reaches or exceeds n w When shrinking the length of the dynamic time window, the minimum time span t must be maintained. wmin ; Step S24: Adaptively adjust the length t of the dynamic time window based on the above steps. w The candidate task pool is updated in real time to ensure that the dynamic time window overlaps with at most two consecutive static time windows.

5. The cluster job planning method based on a dual time window mechanism according to claim 1, characterized in that, In step 3, considering time and resource constraints, a multi-objective optimization model integrating task processing timeliness, resource utilization, and system load balancing is constructed, as follows: in Let be the completion time of task i. Given its expected latest completion time, This is an indicator function; its value is 1 if the condition is true, and 0 otherwise. m,k (t) represents the usage of the k-th type of resource on server m at time t. U represents the total capacity of this resource on server m; m This represents the average resource utilization rate of the m-th computing node. The value represents the average utilization of all nodes; T represents the total time span, and M represents the total number of nodes; the start time of task i is denoted as . The earliest start time of task i is denoted as The set of all its preceding tasks is Pre(i); if task i is assigned to server m for execution, then a im =1, otherwise a im =0.

6. The cluster job planning method based on a dual time window mechanism according to claim 1, characterized in that, In step 4, the cluster job scheduling is modeled as a multi-agent Markov decision process, setting a hierarchical action space for job and server running states, task allocation and execution methods, specifically including: The running status of each job is set, including: whether it is completed, the remaining execution time, the ideal execution time period, whether there are predecessor tasks, whether there are successor tasks, the amount of various resources required, and the execution time required. Setting the server's running status includes: window status information, which describes the discrete distribution characteristics of the server's running status within the first intersecting static time window and the subsequent static time window when dynamic and static time windows overlap. The discrete distribution characteristics include the minimum and maximum values ​​of availability, utilization, and runtime for each type of resource group on the server; the calculation method is as follows: assuming there are K types of computing resources on the server, and the capacity of each type of resource is denoted as R. k The computing units of this type of resource are sorted according to availability, and the sorting results are divided into G sub-resource groups to form dynamic resource groups; the utilization rate and availability rate of resources are calculated in each sub-resource group; the statistical process is carried out independently in the above two static time windows.

7. The cluster job planning method based on a dual time window mechanism according to claim 6, characterized in that, In step 4, the task allocation and execution method adopts hierarchical action space modeling, specifically including: Within each scheduling time step, each server acts as an intelligent agent, selecting a task from its candidate task set and determining the execution method based on its current running state and the state of the selected task. For the selected task, if the j-th execution method is selected, the server selects available computing resources from the resources in the j-th sub-resource group after sorting. Under the premise of satisfying the time constraint, the server determines the start execution time of the task and performs corresponding resource allocation scheduling according to the greedy strategy of earliest completion time first.

8. The cluster job planning method based on a dual time window mechanism according to claim 6, characterized in that, In step 4, the reward function that reflects the scheduling balance is designed, specifically including: Scheduling balance includes task processing timeliness, resource utilization, and load balancing. Each server has a reward function, as shown in the following formula: Where α var α is the coefficient for balanced reward. util η1 is the contribution of static time window 1 to the balanced reward, and η2 is the contribution of static time window 1 to the utilization reward; β is the task timeout penalty coefficient, and t is the utilization reward coefficient. i It is the time required for server i to execute the selected task at time step t, where t is the time required. i,start It is the time when the task execution starts, t i,ddl It is the latest expected deadline for the task to complete; r i,t (s i,t ,a i,t ) is the reward that server i receives at time step t, where s i,t This refers to the local state of server i at time step t, including the running state of server i and the states of all its selectable tasks; a i,t It refers to the task selection made by server i at time step t and the method of executing the task; and These are the static time window 1 balance reward and utilization reward. and These are the static time window 2 balance bonus and utilization bonus, calculated as follows: in It represents the overall resource utilization of server i within a static time window of 1. It is the utilization rate of the k-th computing resource of server i within a static time window of 1. It is the average overall resource utilization of M servers within a static time window of 1. It represents the overall resource utilization of server i within a static time window of 2. It is the utilization rate of the k-th computing resource of server i within a static time window of 2. It is the average overall resource utilization of M servers in static time window 2; The total reward is the arithmetic average of the rewards from M servers:

9. The cluster job planning method based on a dual time window mechanism according to claim 6, characterized in that, In step 5, a conflict-free candidate task set is generated for each server using a Bayesian posterior mechanism, specifically as follows: Step S51: Calculate the matching degree value between each server and each candidate task using a heterogeneous graph neural network and a pointer network based on the Transformer structure; Step S52: Remove the matching degree values ​​from the computation graph of the neural network, and calculate the affinity score of each task to each server using the Bayesian posterior formula. The affinity calculation formula is as follows: Where Pr(i|j) represents the matching degree value of server j in selecting task i, and Pr(j|i) represents affinity, which means the probability that task i belongs to server j. It represents the prior probability that server j will accept the task, and indicates the resource idle rate in the current static time window 1 and static time window 2. Step S53: Based on the matching degree between the server and the task, select candidate tasks for each server using probability sampling. If multiple servers sample the same task, it is considered a conflict. Then, conflict mediation is performed according to the affinity score described in step S52. Only the server-task pairing with the highest affinity is retained, and the task in the pairing is added to the candidate task set of the corresponding server. Step S54: Repeat the sampling and conflict resolution process until the current candidate task queue is divided into multiple non-overlapping candidate task sets, each corresponding to a server, thereby generating a non-overlapping set of server candidate tasks for subsequent task scheduling decisions.

10. A cluster job planning system based on a dual time window mechanism, characterized in that, The system for implementing the method according to any one of claims 1 to 9 comprises: The first module is used to divide the entire job into a multi-level structure with execution order based on the dependencies between subtasks in the job and to construct a directed acyclic graph to form a task topology. The second module is used to construct a dual time window mechanism. The static time window is used to statistically analyze the server's running status within a fixed time period, while the dynamic time window is adaptively adjusted based on the real-time feedback information from the candidate task queue. The third module is used to design a multi-objective optimization function that integrates factors such as task processing timeliness, resource utilization and system load balancing, and introduces time constraints and resource constraints into scheduling decisions to optimize overall scheduling performance. The fourth module is used to model cluster job scheduling as a multi-agent Markov decision process, setting up a hierarchical action space for job and server running status, task allocation and execution methods, and designing a reward function that reflects scheduling balance. The fifth module is used to build a hybrid neural network architecture. It uses graph neural networks to extract the temporal correlation features of tasks, uses heterogeneous graph neural networks to encode the heterogeneous characteristics between tasks and servers, and combines a Transformer-based pointer network to generate intelligent matching scores between tasks and servers. The sixth module applies a multi-agent proximal policy optimization algorithm, which achieves agent-based collaborative optimization through a collaborative mechanism of centralized value function and distributed policy update.

Images