A highly dynamic scheduling method and system for workflow computing tasks

Abstract

The present application provides a highly dynamic scheduling method and system for workflow computing tasks, the method comprising: monitoring task scheduling events; when a task scheduling event is triggered, obtaining a simulation optimization task queue to be scheduled according to the triggering conditions of the triggered task scheduling event, the simulation optimization task queue including multiple workflows and the task requirements of each workflow; obtaining multiple currently available servers and the resource characteristics of each server; constructing a scheduling optimization problem according to the task requirements of each workflow and the resource characteristics of each server, and selecting an optimization algorithm, the scheduling optimization problem including constraints and iteration targets; obtaining an encoding method and a decoding method for the optimization algorithm according to the scheduling optimization problem; using the constraints and iteration targets of the scheduling optimization problem as iteration conditions, and adopting the optimization algorithm, encoding method and decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled.

Description

Technical Field

[0001] The present application relates to the technical field of highly dynamic scheduling of workflow computing tasks, and in particular to a method and system for highly dynamic scheduling of workflow computing tasks. Background Art

[0002] Scientific workflow scheduling, especially CAE (Computer Aided Engineering) simulation optimization scheduling, faces the following difficulties:

[0003] 1) The diversity of computing task attributes. Computing tasks include single-node and multi-node computing, and each task has different levels of difficulty, such as short computing tasks, long computing tasks, and complex resource-consuming tasks. These attributes directly affect the resource requirements and execution time of computing tasks.

[0004] 2) Managing inter-task dependencies: Complex dependencies exist between computational tasks on a composite node and their child nodes, particularly in optimization jobs where a parent node must wait for its child nodes to complete. This creates a cyclic convergence problem, which can be expressed as a cyclic dependency within a directed graph (DG). This further complicates scheduling. Traditional scheduling methods cannot directly handle DG workflows containing cycles.

[0005] 3) Computing resource prediction and dynamic scheduling. CAE-related simulation tasks cannot accurately estimate computing resources, including the remaining running time of the computing task and the available system resources.

[0006] 4) Directed scheduling based on task attributes. Task attributes are limited, and it is necessary to ensure that tasks can be dispatched to servers with resources that can solve the task.

[0007] 5) Multi-user multi-task priority. When multiple users submit simulation tasks, how to determine the task priority and queuing strategy is a key issue.

[0008] Therefore, in order to solve the above problems, it is necessary to develop more intelligent and flexible scheduling methods to improve the efficiency and accuracy of CAE simulation optimization. Summary of the Invention

[0009] The embodiments of the present application provide a method and system for highly dynamic scheduling of workflow computing tasks.

[0010] According to a first aspect of the present application, a method for highly dynamic scheduling of workflow computing tasks is provided, the method comprising:

[0011] Monitor task scheduling events;

[0012] When the task scheduling event is triggered, a simulation optimization task queue to be scheduled is obtained according to the triggering condition of the triggered task scheduling event, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow;

[0013] Get the currently available servers and the resource characteristics of each server;

[0014] Combine the task requirements of each workflow and the resource characteristics of the server to construct a scheduling optimization problem and select an optimization algorithm;

[0015] According to the scheduling optimization problem, obtaining an encoding method and a decoding method for the optimization algorithm;

[0016] The constraint conditions and iteration target of the scheduling optimization problem are used as iteration conditions, and the optimization algorithm, the encoding method and the decoding method are adopted to determine the scheduling method of the simulation optimization task queue to be scheduled.

[0017] According to an embodiment of the present application, the triggering conditions of the task scheduling event include the addition of a new workflow, a change in task status, a fluctuation in resource load, and a decrease in computing performance; accordingly, obtaining a simulation optimization task queue to be scheduled based on the triggering conditions of the triggered task scheduling event includes:

[0018] When the trigger condition is the addition of a new workflow, the new workflow is used as a simulation optimization task queue to be scheduled;

[0019] When the triggering condition is a task state change, resource load fluctuation, or computing performance degradation, currently unscheduled computing tasks are obtained and the currently unscheduled computing tasks are determined as a simulation optimization task queue to be scheduled.

[0020] According to an embodiment of the present application, when the trigger condition is the addition of a new workflow, obtaining the simulation optimization task queue to be scheduled according to the trigger condition of the triggered task scheduling event includes:

[0021] Acquire task flow data that triggers the task scheduling event, wherein the task flow data includes a plurality of directed acyclic graph workflows and / or a plurality of directed cyclic graph workflows;

[0022] In the case where the task process data includes directed cyclic graph workflows, splitting each directed cyclic graph workflow into directed acyclic graph workflows according to the number of task cycles of multiple tasks in each directed cyclic graph workflow;

[0023] Add all directed acyclic graph workflows to the simulation optimization task queue.

[0024] According to an embodiment of the present application, the constraint conditions include a first optimization constraint and a second optimization constraint, the task requirements include the task execution order and task dependencies of each computing task in each workflow, and the resource characteristics include resource limitation information of each server; accordingly,

[0025] The scheduling optimization problem is constructed based on the task requirements of each workflow and the resource characteristics of each server, and the optimization algorithm is selected, including:

[0026] Determine the last computing task of each workflow according to the task execution order of each computing task shown in the task requirements of each workflow, and minimize the task completion time of the last computing task of the currently schedulable workflow as the iteration goal;

[0027] According to the task dependency of each computing task shown in the task requirements of each workflow, determining that the start execution time of each computing task is no earlier than the earliest start time of each computing task as the first optimization constraint, wherein the earliest start time of each computing task is determined according to the completion time of its predecessor computing task;

[0028] According to the resource limitation information shown in the resource characteristics of each server, determining that the task resources of all computing tasks received by each server are not greater than the resource limitation information corresponding to each server as the second optimization constraint;

[0029] An optimization algorithm is determined based on the task requirements of each workflow and the predicted execution time of each workflow. The optimization algorithm includes a task scheduling algorithm, a genetic algorithm, a particle swarm optimization algorithm, a simulated annealing algorithm, and a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm.

[0030] According to an embodiment of the present application, the scheduling method includes the scheduling server and execution order of each computing task of each workflow in the simulation optimization task queue to be scheduled; accordingly,

[0031] Taking the constraints and iteration target of the scheduling optimization problem as iteration conditions, the optimization algorithm, the encoding method, and the decoding method are used to determine the scheduling method of the simulation optimization task queue to be scheduled, including:

[0032] Encoding the simulation optimization task queue based on the optimization algorithm and the encoding method to obtain multiple sets of encoding data, each set of encoding data including the corresponding server and execution order of each computing task in the simulation optimization task queue;

[0033] Taking the constraints and iteration targets of the scheduling optimization problem as iteration conditions, each set of encoded data is decoded using the decoding method. When the scheduling optimization problem corresponding to the decoding result meets the set conditions, the iteration is determined to be finished, and the current decoding result is determined as the scheduling method of the simulation optimization task queue to be scheduled.

[0034] According to an embodiment of the present application, the encoding method includes determining the matching priority of each computing task for each server; accordingly,

[0035] The simulation optimization task queue is encoded based on the optimization algorithm and the encoding method to obtain multiple sets of encoded data, including:

[0036] The optimization algorithm is used to determine multiple groups of encoding data for the simulation optimization task queue to be scheduled, each group of encoding data includes the matching priority of each computing task of each workflow in the simulation optimization task queue to be scheduled and each server.

[0037] According to an embodiment of the present application, the decoding method includes:

[0038] Obtain resource status information and task execution status information of each currently available server;

[0039] Determining executable tasks in the simulation optimization task queue according to the task requirements of each workflow in the simulation optimization task queue and the task execution status information;

[0040] Matching each computing task in each set of coded data with an executable computing task in the simulation optimization task queue, updating the matching priority of the computing task that fails to match to a set value, and obtaining first updated coded data for each set of coded data;

[0041] Determining, based on the resource limitation information and resource status information of each server, servers that do not meet the operation requirements in the first updated code data corresponding to each set of code data, and updating the matching priority of the servers that do not meet the operation requirements to the set value, thereby obtaining second updated code data for each set of code data;

[0042] For each set of encoded data, the computing tasks in the simulation optimization task queue are scheduled according to the corresponding second updated encoded data to obtain a decoding result. The decoding result includes the task completion time of the simulation optimization task queue and the execution order and scheduling server of each computing task in the simulation optimization task queue.

[0043] According to an embodiment of the present application, for each set of coded data, according to its corresponding second updated coded data, the tasks in the simulation optimization task queue are scheduled to obtain a decoding result, including:

[0044] For each set of coded data, according to the matching priority between each computing task and each server indicated by the corresponding second updated coded data, each computing task is assigned to the server having the highest matching priority relationship therewith;

[0045] When all computing tasks of the simulation optimization task queue are allocated and completed, the scheduling server and execution order of each computing task are obtained, and the task completion time of the simulation optimization task queue is calculated.

[0046] According to an embodiment of the present application, the calculation simulation optimization task queue task completion time includes:

[0047] Get the server corresponding to the last calculation task of each workflow in the simulation optimization task queue;

[0048] Determine the start time of the last computing task based on the task type of the last computing task and the server corresponding to the last computing task;

[0049] Determine the task end time of the last calculation task according to the start execution time of the last calculation task and the task prediction time of the last calculation task;

[0050] Compare the task end time of the last calculation task of each workflow, and take the maximum value as the task completion time of the simulation optimization task queue.

[0051] According to an embodiment of the present application, when the scheduling optimization problem corresponding to the decoding result meets the set conditions, determining the end of the iteration includes:

[0052] When the task completion time of the simulation optimization task queue shown in the current decoding result meets the iteration target requirements of the scheduling optimization problem, each computing task of each workflow meets the constraints, and each available server meets the constraints, it is determined that the scheduling optimization problem corresponding to the current decoding result meets the set conditions and the iteration is ended.

[0053] According to a second aspect of the present application, a highly dynamic scheduling system for workflow computing tasks is provided, the system comprising:

[0054] Monitoring module, used to monitor task scheduling events;

[0055] A first acquisition module is configured to acquire, when the task scheduling event is triggered, a simulation optimization task queue to be scheduled according to a triggering condition of the triggered task scheduling event, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow;

[0056] The second acquisition module is used to obtain multiple currently available servers and resource characteristics of each server;

[0057] A construction module is used to construct a scheduling optimization problem based on the task requirements of each workflow and the resource characteristics of each server, and select an optimization algorithm. The scheduling optimization problem includes an iteration goal and constraints.

[0058] A third acquisition module is used to acquire an encoding method and a decoding method for the optimization algorithm according to the scheduling optimization problem;

[0059] The determination module is used to use the constraint conditions and iteration targets of the scheduling optimization problem as iteration conditions, and adopt the optimization algorithm, the encoding method and the decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled.

[0060] The method of an embodiment of the present application monitors task scheduling events; when the task scheduling event is triggered, obtains a simulation optimization task queue to be scheduled according to the triggering conditions of the triggered task scheduling event, and the simulation optimization task queue includes multiple workflows and the task requirements of each workflow; obtains multiple currently available servers and the resource characteristics of each server; constructs a scheduling optimization problem according to the task requirements of each workflow and the resource characteristics of each server, and selects an optimization algorithm; according to the scheduling optimization problem, obtains an encoding method and a decoding method for the optimization algorithm; uses the constraints and iteration targets of the scheduling optimization problem as iteration conditions, and adopts the optimization algorithm, the encoding method and the decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled. A dynamic monitoring mechanism is used to capture task scheduling requirements, comprehensively collect and analyze simulation optimization task queues and server resource information, accurately construct scheduling optimization problems, adaptively select optimization algorithms, and combine specific encoding and decoding methods to achieve intelligent iterative solution of task scheduling, improve the flexibility and accuracy of scheduling decisions, and significantly enhance scheduling efficiency in complex situations such as large task time spans and unpredictable resource consumption, ensuring efficient execution of workflows, thereby solving the problem that traditional scheduling methods cannot schedule CAE simulation optimization tasks.

[0061] It should be understood that the teachings of this application do not necessarily achieve all of the beneficial effects described above, but that specific technical solutions can achieve specific technical effects, and other embodiments of this application can also achieve beneficial effects not mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an illustrative and non-limiting manner, in which:

[0063] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0064] Figure 1 The following is a flowchart illustrating a highly dynamic scheduling method for workflow computing tasks according to an embodiment of the present application;

[0065] Figure 2 A schematic diagram illustrating the implementation flow of the scheduling optimization problem and the optimization algorithm determination operation of the highly dynamic scheduling method for workflow computing tasks provided by an embodiment of the present application is shown;

[0066] Figure 3 A schematic diagram illustrating an implementation flow of a task scheduling mode determination operation of a highly dynamic scheduling method for workflow computing tasks provided by an embodiment of the present application is shown;

[0067] Figure 4 A schematic diagram of the implementation flow of the decoding operation of the highly dynamic scheduling method for workflow computing tasks provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0068] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0069] Figure 1 The following is a flowchart showing an implementation of a highly dynamic scheduling method for workflow computing tasks provided in an embodiment of the present application.

[0070] refer to Figure 1 , an embodiment of the present application provides a highly dynamic scheduling method for workflow computing tasks, the method comprising:

[0071] Operation 101: monitoring task scheduling events.

[0072] In one embodiment of the present application, in order to monitor the task, a task scheduling event is configured for the task in advance, so as to determine whether the task needs to be scheduled by monitoring whether the task scheduling event is triggered.

[0073] Operation 102 : When a task scheduling event is triggered, a simulation optimization task queue to be scheduled is obtained according to a triggering condition of the triggered task scheduling event. The simulation optimization task queue includes multiple workflows and task requirements of each workflow.

[0074] In one embodiment of the present application, the triggering conditions of the task scheduling event include the addition of a new workflow, a change in task status, a fluctuation in resource load, and a decrease in computing performance; accordingly, the simulation optimization task queue to be scheduled is obtained according to the triggering conditions of the triggered task scheduling event, including: when the triggering condition is the addition of a new workflow, the new workflow is used as the simulation optimization task queue to be scheduled; when the triggering condition is a change in task status, a fluctuation in resource load, or a decrease in computing performance, the currently unscheduled task is obtained, and the currently unscheduled task is determined as the simulation optimization task queue to be scheduled.

[0075] Tasks need to be rescheduled when new workflows are added, task status changes occur, resource load fluctuates, or performance degrades. Therefore, you can configure trigger conditions for task scheduling events, including new workflow additions, task status changes, resource load fluctuations, and computing performance degradation. This triggers the task scheduling event and initiates task scheduling when new workflows are added, task status changes occur, resource load fluctuates, or performance degrades occur.

[0076] When a new workflow is added, it can be directly assigned to the simulation optimization task queue to be scheduled. However, when a task status changes, resource load fluctuates, or performance degrades, the task is still being scheduled. Unscheduled tasks are then added to the simulation optimization task queue to be scheduled. To improve scheduling efficiency, already scheduled tasks are not rescheduled.

[0077] In one embodiment of the present application, when the trigger condition is the addition of a new workflow, the simulation optimization task queue to be scheduled is obtained according to the trigger condition of the triggered task scheduling event, including: obtaining the task flow data that triggers the task scheduling event, the task flow data including multiple directed acyclic graph workflows and / or multiple directed cyclic graph workflows; when the task flow data includes directed cyclic graph workflows, each directed cyclic graph workflow is split into directed acyclic graph workflows according to the number of task cycles of multiple tasks in each directed cyclic graph workflow; and all directed acyclic graph workflows are added to the simulation optimization task queue.

[0078] The workflow of the present application may include directed cyclic graph workflows and directed acyclic graph workflows. When the workflow type is a directed cyclic graph workflow, the directed cyclic graph workflow needs to be decomposed into multiple directed acyclic graph workflows. When the workflow is a directed acyclic graph workflow, no processing is performed. When all directed cyclic graph workflows are decomposed, all directed acyclic graph workflows are determined to be simulation optimization task queues to be scheduled.

[0079] Directed cyclic graph workflows can be divided into two types: the case where the number of task iterations is known and the case where the number of task iterations is unknown. In the known case, each task requiring iteration is replicated according to its iteration number, forming multiple tasks. The replication follows the dependencies of the directed cyclic graph workflow, resulting in multiple directed acyclic graph workflows. In the unknown case, where the iteration number depends on real-time data or resource changes, an iterative DAG (Directed Acyclic Graph) generation method is used to split the directed cyclic graph workflow into multiple directed acyclic graph workflows.

[0080] In one embodiment of the present application, multiple workflows in the simulation optimization task queue can be submitted simultaneously by multiple users. The multiple workflows include multiple computing tasks with dependencies and task execution order, and the computing tasks are CAE simulation optimization tasks.

[0081] Operation 103: Acquire multiple currently available servers and resource characteristics of each server.

[0082] To support computing task scheduling, multiple servers are typically configured and scheduled across these servers. However, not all configured servers are always available when scheduling is required. Therefore, when scheduling is required, it is necessary to obtain currently available servers and their resource characteristics. Server resource characteristics can include information such as the server's CPU (Central Processing Unit) and RAM (Random Access Memory).

[0083] In one embodiment of the present application, the task scheduling of the present application supports traditional distributed computing (private deployment) and public cloud. Therefore, the multiple servers can be private deployment servers or public cloud servers. The multiple servers can also include both private deployment servers and public cloud servers.

[0084] Operation 104 : construct a scheduling optimization problem based on the task requirements of each workflow and the resource characteristics of each server, and select an optimization algorithm. The scheduling optimization problem includes an iteration target and constraints.

[0085] The task requirements usually show the total number of tasks and the task type of each computing task. The resource characteristics of the server can show information such as available resources and resource stability. The complexity of the computing task can be determined based on the task type, and the scale of the computing task can be determined based on the total number of tasks.

[0086] By analyzing task size, task complexity, and available resources, we can construct scheduling optimization problems suitable for all workflows, such as maximizing resource utilization and minimizing completion time. Scheduling optimization problems include constraints and iteration goals.

[0087] Different optimization algorithms can be configured in advance for different task sizes, complexities, and available resources, such as task scheduling algorithms, genetic algorithms, particle swarm optimization, simulated annealing, or hybrid optimization algorithms. Once the simulated optimization task queue is obtained, the corresponding optimization algorithm can be matched based on the task size, complexity, and available server resources.

[0088] In one embodiment of the present application, a scheduling optimization problem is constructed and an optimization algorithm is selected by combining the task requirements of each workflow and the resource characteristics of the server. The task requirements of each workflow and the resource characteristics of each server can also be provided to the user, and the scheduling optimization problem and optimization algorithm provided by the user can be received.

[0089] Operation 105 : Obtain an encoding method and a decoding method for the optimization algorithm according to the scheduling optimization problem.

[0090] The encoding and decoding methods of the optimization algorithm are determined in order to convert the discrete scheduling optimization problem into decision variables that can be recognized by the optimization algorithm through the encoding and decoding methods, so as to couple the optimization algorithm and the scheduling optimization problem.

[0091] The scheduling optimization problem shows the goal of this scheduling task, such as maximizing resource utilization and minimizing completion time. According to different scheduling optimization problems, encoding and decoding methods for different optimization algorithms are configured in advance. After obtaining the scheduling optimization problem, the encoding and decoding methods that match it can be obtained.

[0092] Operation 106 , using the constraints and iteration target of the scheduling optimization problem as iteration conditions, and adopting the optimization algorithm, encoding method, and decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled.

[0093] After determining the encoding and decoding methods, the constraints and iteration targets of the scheduling optimization problem are used as the iteration conditions of the optimization algorithm. Based on the optimization algorithm and encoding method, the simulation optimization task queue is continuously encoded and decoded. Once the scheduling optimization problem is iteratively completed, i.e., the algorithm converges, the iteration is considered complete. The decoding result of the completed iteration is the scheduling method for the simulation optimization task queue to be scheduled. The scheduling method generally includes the execution order of each computing task in the simulation optimization task queue and the scheduling server.

[0094] Therefore, the embodiment of the present application utilizes a dynamic monitoring mechanism to capture task scheduling requirements, comprehensively collects and analyzes simulation optimization task queues and server resource information, accurately constructs scheduling optimization problems, and appropriately selects optimization algorithms, combined with specific encoding and decoding methods, to achieve intelligent iterative solution of task scheduling, improve the flexibility and accuracy of scheduling decisions, significantly enhance scheduling efficiency in complex situations such as large task time spans and unpredictable resource consumption, ensure efficient execution of workflows, and thus solve the problem that traditional scheduling methods cannot schedule CAE simulation optimization tasks.

[0095] Figure 2 A schematic diagram of the implementation flow of the scheduling optimization problem and optimization algorithm determination operation of the highly dynamic scheduling method for workflow computing tasks provided by an embodiment of the present application is shown.

[0096] refer to Figure 2 In one embodiment of the present application, the constraints include a first optimization constraint and a second optimization constraint, the task requirements include the task execution order and task dependencies of each computing task in each workflow, and the resource characteristics include resource limitation information of each server. Accordingly, the above operation 104 determines the scheduling optimization problem and the optimization algorithm based on the task requirements of each workflow and the resource characteristics of each server, including:

[0097] Operation 201 , determining the last computing task of each workflow according to the task execution order of each computing task shown in the task requirements of each workflow, and minimizing the task completion time of the last computing task of the currently schedulable workflow as an iteration goal;

[0098] Operation 202: Based on the task dependency of each computing task shown in the task requirements of each workflow, determine that the start execution time of each computing task is no earlier than the earliest start time of each computing task as a first optimization constraint, and the earliest start time of each computing task is determined based on the completion time of its predecessor computing task;

[0099] Operation 203 , determining, based on the resource limitation information shown in the resource characteristics of each server, that the task resources of all computing tasks received by each server are not greater than the resource limitation information corresponding to each server as a second optimization constraint;

[0100] Operation 204 : determining an optimization algorithm based on the task requirements of each workflow and the predicted execution time of each workflow. The optimization algorithm includes a task scheduling algorithm, a genetic algorithm, a particle swarm optimization algorithm, a simulated annealing algorithm, and a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm.

[0101] The scheduling optimization problem mainly includes iterative objectives and constraints. The iterative objectives are the optimization objectives, and the constraints are the constraints that need to be met, including the first optimization constraint and the second optimization constraint.

[0102] To ensure scheduling efficiency, the iteration objective of the scheduling optimization problem is typically configured to minimize the completion time of the simulation optimization task queue. Because workflows may have execution order issues due to dependencies between tasks, the shortest completion time of the simulation optimization task queue should be the shortest completion time of the currently schedulable workflows. The completion time of the currently schedulable workflow can be understood as the end time of its last computational task. Therefore, minimizing the end time of the last computational task of the currently schedulable workflow can be set as the iteration objective.

[0103] For example, the task end time of the last computation task of the current schedulable workflow can be ,in, Indicates the last calculation task, w n Indicates the n Currently schedulable workflows, Indicates the last calculation task The server instance, FT Indicates the final completion time of the workflow. The iteration goal can be ( W )),in, W For a set of workflows, the iteration goal can be understood as minimizing the task completion time of the last computation task in the current schedulable workflow.

[0104] The task execution process usually involves task logic constraints and resource constraints, namely the first optimization constraint and the second optimization constraint. In order to ensure that the tasks are executed in the task execution order shown by the task requirements, the first optimization constraint needs to be configured so that the start execution time of each computing task is not earlier than the earliest start time of each computing task. Among them, the earliest start time of each computing task can be determined based on the completion time of its predecessor computing task. The predecessor computing task of each computing task is the task that has a logical relationship with it and is executed before it.

[0105] To adapt task scheduling to server resources, a second optimization constraint is configured: the task resources for all computing tasks received by each server must not exceed the resource limit information corresponding to each server. Each server is configured with instances for different task types, and the task resources for all computing tasks received by each server can be considered the task resources required for all instances on each server. The resource limit information can be the available resources of the server.

[0106] For example, kThe task resources required for each task type can be r k , for the i servers y i All instances of I kij The total amount of resources required can be expressed as , then the second constraint is ,in, I kij Indicates that on each server y i The deployed j indivual s k An instance of a task type.

[0107] After formulating the scheduling optimization problem, we determine an optimization algorithm based on the task requirements of each workflow and its predicted execution time. These optimization algorithms include task scheduling algorithms, genetic algorithms, particle swarm optimization, simulated annealing algorithms, and hybrid optimization algorithms based on heuristic algorithms and evolutionary algorithms.

[0108] When configuring an optimization algorithm, consider not only task requirements but also the predicted execution time. For example, if the task types are relatively simple (e.g., fewer than 10), the task scale is small (e.g., less than 500 tasks), and the predicted execution time is short (e.g., less than 1 hour), you can configure a corresponding classic scheduling algorithm, such as the Min-Min or Max-Min task scheduling algorithm. If the task scale is large, the complexity is high (multiple task types), and the predicted execution time is long, you can configure the optimization algorithm to be a genetic algorithm, particle swarm optimization, simulated annealing, or a hybrid optimization algorithm based on heuristic and evolutionary algorithms.

[0109] Therefore, after determining the total number of tasks and the number of task types based on task requirements, and determining the total task predicted execution time based on the predicted execution time of each workflow, the corresponding optimization algorithm can be matched through analysis.

[0110] Figure 3 A schematic diagram of the implementation flow of the task scheduling mode determination operation of the highly dynamic scheduling method for workflow computing tasks provided by an embodiment of the present application is shown.

[0111] refer to Figure 3In one embodiment of the present application, the scheduling method includes the scheduling server and execution order of each computing task of each workflow in the simulation optimization task queue to be scheduled; accordingly, the above operation 106 uses the constraints and iteration target of the scheduling optimization problem as iteration conditions, adopts the optimization algorithm, encoding method and decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled, and includes:

[0112] Operation 301 encodes the simulation optimization task queue based on the optimization algorithm and encoding method to obtain multiple sets of encoded data, each set of encoded data including the server and execution order corresponding to each computing task in the simulation optimization task queue;

[0113] Operation 302 uses the constraints and iteration targets of the scheduling optimization problem as iteration conditions, and decodes each set of encoded data using a decoding method. When the scheduling optimization problem corresponding to the decoding result meets the set conditions, the iteration is determined to be complete, and the current decoding result is determined as the scheduling method of the simulation optimization task queue to be scheduled.

[0114] Encoding is the coupling point between the algorithm and the problem being optimized, defining the set of decision variables that the algorithm can understand. For workflow task scheduling, the optimization algorithm uses encoding to generate a representation of the scheduling servers and execution order for each computational task in the simulation optimization task queue. To meet optimization requirements, the simulation optimization task queue must be encoded multiple times, generating multiple sets of encoded data containing task scheduling information.

[0115] The encoding is understandable to the optimization algorithm and represents the solution space for the task scheduling problem. Decoding converts the encoding results into an intuitive and understandable execution sequence and method. During the optimization process, each encoding only needs to be decoded once to generate the corresponding decoding result. The optimization algorithm continuously generates new encodings, decodes each encoding, and evaluates the quality of the corresponding solution, thereby gradually searching for the possible optimal solution.

[0116] In one embodiment of the present application, the condition for terminating an iteration is that the scheduling solution generated by the decoding result satisfies the set conditions of the scheduling optimization problem, namely, the task completion time of the simulation optimization task queue corresponding to the current decoding result meets the iteration target requirements, each computing task in each workflow satisfies the predetermined constraints, and each available server satisfies the predetermined constraints. If all of the above conditions are met, it is determined that the scheduling optimization problem corresponding to the current decoding result meets the set conditions, and the iteration is terminated.

[0117] The iteration objective of the scheduling optimization problem is to minimize the completion time of the last computational task of the currently schedulable workflow, that is, to minimize the task completion time of the simulation optimization task queue. Therefore, when decoding each set of encoded data, if the decoding result shows that the completion time of the last workflow is minimized, it can be considered that the task completion time of the simulation optimization task queue meets the iteration objective. In addition to meeting the iteration objective, each computational task and each server must also meet the constraints. Only then is the scheduling optimization problem corresponding to the decoding result considered to meet the set conditions and the iteration of the optimization algorithm terminated.

[0118] In one embodiment of the present application, the encoding method includes determining the matching priority of each computing task for each server; accordingly, the simulation optimization task queue is encoded based on the optimization algorithm and the decoding method to obtain multiple sets of encoded data, including: using the optimization algorithm to determine multiple sets of encoded data for the simulation optimization task queue to be scheduled, each set of encoded data includes the matching priority of each computing task of each workflow in the simulation optimization task queue to be scheduled and each server.

[0119] The optimization algorithm calculates the matching priorities between the workflow tasks in the simulation optimization task queue and each server, generating multiple sets of encoded data. The matching priorities are encoded to indicate the degree of compatibility between the tasks and servers. Decoding directly yields the specific task-server combinations.

[0120] For example, the encoding format can be: X ={ p wky | w ∈ W , k ∈ S w , y ∈ Y},in, X To match the priority set, p wky Represents a computational workflow w No. k Tasks for the server Y The matching priority. W Represents the workflow set of the simulation optimization task queue to be scheduled, S w Represents a workflow w The set of all tasks, Y Represents the set of available servers. Based on the optimization algorithm combined with the above encoding method, the simulation optimization task queue can be encoded, and the optimal matching priority set X can be calculated through the optimization algorithm.

[0121] Figure 4A schematic diagram of the implementation flow of the decoding operation of the highly dynamic scheduling method for workflow computing tasks provided by an embodiment of the present application is shown.

[0122] refer to Figure 4 In one embodiment of the present application, the decoding method includes:

[0123] Operation 401: Obtain resource status information and task execution status information of each currently available server;

[0124] Operation 402 , determining executable computing tasks in the simulation optimization task queue based on the task requirements and task execution status information of each workflow in the simulation optimization task queue;

[0125] Operation 403: Match each computing task in each set of coded data with an executable computing task in the simulation optimization task queue, and update the matching priority of the computing task that fails to match to a set value, thereby obtaining first updated coded data for each set of coded data;

[0126] Operation 404: determining servers that do not meet the operating requirements in the first updated encoding data corresponding to each set of encoding data based on the resource limitation information and resource status information of each server, updating the matching priority of the servers that do not meet the operating requirements to a set value, and obtaining second updated encoding data for each set of encoding data;

[0127] Operation 405, for each set of encoded data, schedule the computing tasks in the simulation optimization task queue according to its corresponding second updated encoded data to obtain a decoding result, which includes the task completion time of the simulation optimization task queue and the execution order and scheduling server of each computing task in the simulation optimization task queue.

[0128] Each set of encoded data includes all servers corresponding to each computing task in the simulation optimization task queue and the matching priority of each server. However, the task scheduling process is ongoing, and accordingly, the server resource usage is also constantly changing. Therefore, it is necessary to obtain the current actual task execution status information and resource status information. Among them, the task execution status information is used to indicate whether the task has been executed, and the resource status information is used to indicate the resource usage of the server.

[0129] After obtaining the task execution status information and resource status information, the currently executable tasks in the simulation optimization task queue (i.e., tasks whose predecessor tasks have completed but have not yet been executed) can be screened using the task execution status information and the task dependencies shown in each workflow's task requirements. The tasks in each set of coded data are then matched with the currently executable tasks. If no match is found, the unmatched tasks are no longer required to participate in this iteration. The matching priority corresponding to the unmatched tasks in each set of coded data is configured to a set value, thereby obtaining the first updated coded data corresponding to each set of coded data.

[0130] After filtering out the tasks that do not meet the requirements, it is also necessary to filter out the servers that do not meet the requirements. That is, for each computing task in the first updated coding data corresponding to each set of coding data, based on the resource limitation information and resource status information of each server, determine whether the resources of the server with a matching priority relationship with each computing task can support the execution of the corresponding task. If it cannot be supported, the matching priority between the task and the server is set to a set value to obtain the second updated coding data.

[0131] In one embodiment of the present application, the set value may be infinite.

[0132] After updating each set of coded data, the second updated coded data is used to dynamically allocate each computing task in the simulation optimization task queue to achieve scheduling and obtain a decoding result. The decoding result can be regarded as a scheduling method, including the scheduling server and execution order of each computing task in the simulation optimization task queue.

[0133] In one embodiment of the present application, for each group of encoded data, tasks in the simulation optimization task queue are scheduled according to the corresponding second updated encoded data to obtain a decoding result, including: for each group of encoded data, according to the matching priority of each computing task and each server shown by the corresponding second updated encoded data, each computing task is assigned to the server with the highest matching priority (lowest numerical value) relationship with it; when all tasks in the simulation optimization task queue are assigned, the scheduling server and execution order of each computing task are obtained, and the task completion time of the simulation optimization task queue is calculated.

[0134] After determining the second updated coding data, the computing task, workflow and server combination with the highest matching priority (i.e., the earliest execution start or the lowest resource consumption) can be found among all currently executable computing tasks and server combinations according to the second updated coding data, and the computing tasks can be allocated based on the combination. After that, all computing tasks can be allocated by continuously repeating the above process of determining the computing task, workflow and server combination with the highest matching priority and allocating them based on the combination, thereby completing the allocation of all computing tasks.

[0135] After all computing tasks are assigned, the decoding result for each set of encoded data can be obtained by obtaining the server to which each computing task is assigned and the execution order.

[0136] After obtaining the decoding results, it is also necessary to calculate the task completion time of the simulation optimization task queue.

[0137] In one embodiment of the present application, the task completion time of the simulation optimization task queue is calculated, including: obtaining the server corresponding to the last computing task of each workflow in the simulation optimization task queue; determining the start execution time of the last computing task based on the task type of the last computing task and the server corresponding to the last computing task; determining the task end time of the last computing task based on the start execution time of the last computing task and the task prediction time of the last computing task; comparing the task end time of the last computing task of each workflow, and taking the maximum value as the task completion time of the simulation optimization task queue.

[0138] The multiple servers currently available can be viewed as a collection of server resources: Y ={ y 1,……, y h}, h is the total number of available servers. The resource characteristics of each server include resource limitation information, which can be expressed as y i =[ cpu i , ram i ], cpu i Indicates the i CPU information of each server, ram i Indicates the i RAM information of each server.

[0139] Tasks usually have m Task types, the set of task types can be S ={ S 1,……, Sm Each computing task type can be deployed as an instance, and each computing task type has different task requirements. r =[ r 1,……, r m ] 2xm , then each type of task S m The requirements for CPU and RAM can be expressed as r m =[ cpu m , ram m ] T Among them, each server can deploy one instance or multiple instances, that is, a server can support one task type or multiple task types. I kij Indicates that the server i The deployed j indivual S k An instance of a task type.

[0140] The tasks are shown in the task requirements of each workflow s t The dependency relationship between them, that is, the task dependency relationship of each computing task, the dependency set between tasks can be represented by E Indicates that the workflow can be w Indicates that multiple workflows w Forming a workflow collection: W ={ w 1, w 2…… w n}. ,in, e pk Indicates dependency relationships, such as e pk =0, it means the task and There is no data dependency between e pk ≠0, then its value represents the task Towards The amount of data transferred.

[0141] Therefore, each computation task The completion time of each computation task can be considered as In the instance I kij The completion time (FT) on FT ( , I kij )= ST ( , I kij )+ ET ( ),in, ST ( , I kij ) refers to the task In the instance I kij The time when the task starts execution, that is, the At the start execution time of the corresponding server, ET ( ) refers to the task Predict the time it takes to complete the task The task prediction time can be estimated based on the task type and historical data.

[0142] Among them, the task At the start time of the corresponding server ST( ，I kij ) Constraints need to be met ST( ， I kij )≥EST( ，I kij ) , EST( ，I kij ) It's a task In the instance I kij The earliest start time (EarliestStartTime, EST) on the task The earliest completion time on the server, determined by the instance I kij The available time is determined by EST( ，I kij )=max{FT( ，I pi'j' )} , ∈ pred(Ipi'j' ) , i' It is to carry out the task Server, j' yes Belongs to j' instances.

[0143] By using the above method to determine task completion time, we can obtain the task completion time of each workflow by obtaining the task completion time of the last calculation task. We can also obtain the task completion time of the simulation optimization task queue by taking the maximum task completion time of each workflow.

[0144] For example, the decoding process in the embodiment of the present application may include:

[0145] S1, obtain the server resource status and simulation optimization task queue.

[0146] Obtain resource status information (resource status) of available servers and the queue of simulation optimization tasks to be scheduled.

[0147] S2, filter out the currently computable task set of each workflow .

[0148] According to the task execution order and task dependency shown in the task requirements of each workflow, all tasks whose predecessor tasks have been completed are determined to be currently computable tasks. ,in, w Represents a workflow.

[0149] S3, order ,right .

[0150] in, Indicates initialization of a set of encoded data As the encoded data required for the current decoding, Represents a workflow set consisting of multiple workflows in the simulation optimization task queue to be scheduled Any workflow in w , Indicates that for tasks that are not currently computable Tasks in k , Indicates that it will be matched with priority Set to infinity (inf). The purpose is to prevent these tasks or server combinations from being considered in the current iteration.

[0151] S4, yes , Y , determine the server Whether the subtask is satisfied If the operating requirements are not met, .

[0152] For each workflow w The current computable task in Tasks in k , check the server y Whether the task is met k The running requirements (such as CPU, memory, software dependencies, etc.) are met. If they are not met, the task k On the server y The match priority on is set to infinity (inf).

[0153] S5, .

[0154] Among all currently computable task and server combinations, find the task with the lowest priority value (i.e., the earliest to start execution or the best resource consumption) k' , Workflow w' and server y' combination of .

[0155] S6, the workflow No. Tasks are assigned to the server , calculate the current task The earliest start time and end time are updated, and the server resource status and simulation optimization task queue are updated.

[0156] S7, check whether all tasks have been assigned. If so, go to step S8, otherwise go to step S2, and loop through S2-S6 until all tasks are assigned.

[0157] S8, calculates the time required for all tasks to be completed.

[0158] In one embodiment of the present application, the optimization algorithm is a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm. The constraints and iteration targets of the scheduling optimization problem are used as iteration conditions. Each set of encoded data is decoded in a decoding manner. When the scheduling optimization problem corresponding to the decoding result meets the set conditions, the iteration is determined to be complete, and the current decoding result is determined as the scheduling method of the simulation optimization task queue to be scheduled, including:

[0159] 1) Initialization.

[0160] Set the parameters of the evolutionary algorithm, such as the population size Np , crossover probability C , mutation factor F wait.

[0161] Generate an initial population, each individual in the population represents a candidate solution x , that is, randomly generate multiple sets of coded data p wky , .

[0162] 2) Evaluate the population.

[0163] Calculate the fitness value of each individual in the initial population.

[0164] 3) Iterative evolution.

[0165] Perform mutation operation: For each individual in the population, randomly select three different individuals from the population. Generate mutation vector: ,in, x r1 , x r2 , x r3 are three randomly selected individuals, F is the factor of variation.

[0166] Perform crossover operation: for each individual x i and its mutation vector v i Perform crossover operation to generate experimental individuals u i , if the random number is less than the crossover probability or j is a randomly selected dimension, then u i (j)=v i (j) ,otherwise u i (j)=x i (j) .

[0167] Perform selection operation: Calculate the experimental individuals u i The fitness value of u i The fitness value is better than x i , then use u i replace x i , otherwise keep x i .

[0168] 4) Neighborhood search (performed after each fixed-interval iteration step).

[0169] Perform a field search on some or all individuals in the current population. The field search steps include: 1. Select an individual x i As the initial solution; 2. x i Generate several neighborhood solutions in the neighborhood of x i ' (usually generated by random perturbations in each dimension); 3. Calculate the fitness values of these neighborhood solutions; 4. If a neighborhood solution is found x i ' has a better fitness value than x i , then use x i 'replace x i .

[0170] 5) Termination conditions.

[0171] Determine whether the termination conditions are met. If the maximum number of iterations is reached or the fitness value is no longer significantly improved, the above Figure 4 The decoding method described in the description calculates the task completion time of the simulation optimization task queue.

[0172] If the task completion time of the simulation optimization task queue reaches the shortest, the termination condition is determined to be met, the algorithm ends, and the optimal solution is output. x If the termination condition is not met, return to step 3) and continue iterating.

[0173] 6) Output the final scheduling method.

[0174] The final scheduling method is the optimal solution x The corresponding decoding result. According to the optimal solution x and the above Figure 4 The decoding method determines the task completion time of the simulation optimization task queue, the execution order of each computing task in the simulation optimization task queue and the scheduling server.

[0175] Therefore, the present application combines the global search capability of the evolutionary algorithm and the local optimization capability of the domain search to effectively explore the solution space and improve scheduling efficiency.

[0176] Based on the above method, an embodiment of the present application also provides a highly dynamic scheduling system for workflow computing tasks, which includes: a monitoring module for monitoring task scheduling events; a first acquisition module for obtaining a simulation optimization task queue to be scheduled according to the triggering conditions of the triggered task scheduling event when a task scheduling event is triggered, and the simulation optimization task queue includes multiple workflows and the task requirements of each workflow; a second acquisition module for obtaining multiple currently available servers and the resource characteristics of each server; a construction module for constructing a scheduling optimization problem and selecting an optimization algorithm based on the task requirements of each workflow and the resource characteristics of each server, and the scheduling optimization problem includes iteration objectives and constraints; a third acquisition module for obtaining an encoding method and a decoding method for the optimization algorithm based on the scheduling optimization problem; a determination module for using the constraints and iteration objectives of the scheduling optimization problem as iteration conditions, and adopting an optimization algorithm, an encoding method and a decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled.

[0177] It should be noted that the description of the highly dynamic scheduling system for workflow computing tasks in the embodiment of the present application is similar to the description of the above-mentioned method embodiment, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figures 1 to 4 The present invention shall be understood by reference to the description of any of the accompanying drawings.

[0178] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0179] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A highly dynamic scheduling method for workflow computing tasks, characterized in that: The method comprises: Monitor task scheduling events; When the task scheduling event is triggered, a simulation optimization task queue to be scheduled is obtained according to the triggering condition of the triggered task scheduling event, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow; Get the currently available servers and the resource characteristics of each server; Based on the task requirements of each workflow and the resource characteristics of each server, a scheduling optimization problem is constructed and an optimization algorithm is selected. The scheduling optimization problem includes an iteration objective and constraints. According to the scheduling optimization problem, obtaining an encoding method and a decoding method for the optimization algorithm; Taking the constraints and iteration target of the scheduling optimization problem as iteration conditions, the optimization algorithm, the encoding method and the decoding method are used to determine the scheduling method of the simulation optimization task queue to be scheduled; The triggering conditions of the task scheduling event include the addition of a new workflow, a task status change, resource load fluctuations, and a decrease in computing performance; accordingly, The step of obtaining a simulation optimization task queue to be scheduled according to a triggering condition of a triggered task scheduling event includes: When the trigger condition is the addition of a new workflow, the new workflow is used as a simulation optimization task queue to be scheduled; When the triggering condition is a task state change, resource load fluctuation, or computing performance degradation, currently unscheduled computing tasks are obtained and the currently unscheduled computing tasks are determined as a simulation optimization task queue to be scheduled.

2. The method according to claim 1, characterized in that In the case where the trigger condition is the addition of a new workflow, the step of obtaining the simulation optimization task queue to be scheduled according to the trigger condition of the triggered task scheduling event includes: Acquire task flow data that triggers the task scheduling event, wherein the task flow data includes a plurality of directed acyclic graph workflows and / or a plurality of directed cyclic graph workflows; In the case where the task process data includes directed cyclic graph workflows, splitting each directed cyclic graph workflow into directed acyclic graph workflows according to the number of task cycles of multiple tasks in each directed cyclic graph workflow; Add all directed acyclic graph workflows to the simulation optimization task queue.

3. The method according to claim 1, wherein the constraints include a first optimization constraint and a second optimization constraint, the task requirements include a task execution order and task dependencies of each computing task in each workflow, and the resource characteristics include resource limitation information of each server; accordingly, The scheduling optimization problem is constructed based on the task requirements of each workflow and the resource characteristics of each server, and the optimization algorithm is selected, including: Determine the last computing task of each workflow according to the task execution order of each computing task shown in the task requirements of each workflow, and minimize the task completion time of the last computing task of the currently schedulable workflow as the iteration goal; According to the task dependency of each computing task shown in the task requirements of each workflow, determining that the start execution time of each computing task is no earlier than the earliest start time of each computing task as the first optimization constraint, wherein the earliest start time of each computing task is determined according to the completion time of its predecessor computing task; According to the resource limitation information shown in the resource characteristics of each server, determining that the task resources of all computing tasks received by each server are not greater than the resource limitation information corresponding to each server as the second optimization constraint; An optimization algorithm is determined based on the task requirements of each workflow and the predicted execution time of each workflow. The optimization algorithm includes a task scheduling algorithm, a genetic algorithm, a particle swarm optimization algorithm, a simulated annealing algorithm, and a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm.

4. The method according to claim 1, wherein The scheduling method includes the scheduling server and execution order of each computing task of each workflow in the simulation optimization task queue to be scheduled; accordingly, Taking the constraints and iteration target of the scheduling optimization problem as iteration conditions, the optimization algorithm, the encoding method, and the decoding method are used to determine the scheduling method of the simulation optimization task queue to be scheduled, including: Encoding the simulation optimization task queue based on the optimization algorithm and the encoding method to obtain multiple sets of encoding data, each set of encoding data including the server and execution order corresponding to each computing task in the simulation optimization task queue; Taking the constraints and iteration targets of the scheduling optimization problem as iteration conditions, each set of encoded data is decoded using the decoding method. When the scheduling optimization problem corresponding to the decoding result meets the set conditions, the iteration is determined to be finished, and the current decoding result is determined as the scheduling method of the simulation optimization task queue to be scheduled.

5. The method according to claim 4, characterized in that The encoding method includes determining the matching priority of each computing task for each server; accordingly, The simulation optimization task queue is encoded based on the optimization algorithm and the encoding method to obtain multiple sets of encoding data, including: The optimization algorithm is used to determine multiple groups of encoding data for the simulation optimization task queue to be scheduled, each group of encoding data includes the matching priority of each computing task of each workflow in the simulation optimization task queue to be scheduled and each server.

6. The method according to claim 5, characterized in that The decoding method includes: Obtain resource status information and task execution status information of each currently available server; Determining executable computing tasks in the simulation optimization task queue according to the task requirements of each workflow in the simulation optimization task queue and the task execution status information; Matching each computing task in each set of coded data with an executable computing task in the simulation optimization task queue, updating the matching priority of the computing task that fails to match to a set value, and obtaining first updated coded data for each set of coded data; Determining, based on the resource limitation information and resource status information of each server, servers that do not meet the operating requirements in the first updated encoding data corresponding to each set of encoding data, and updating the matching priority of the servers that do not meet the operating requirements to the set value, thereby obtaining second updated encoding data for each set of encoding data; For each set of encoded data, the computing tasks in the simulation optimization task queue are scheduled according to the corresponding second updated encoded data to obtain a decoding result. The decoding result includes the task completion time of the simulation optimization task queue and the execution order and scheduling server of each computing task in the simulation optimization task queue.

7. The method according to claim 6, characterized in that For each set of coded data, scheduling the computing tasks in the simulation optimization task queue according to the corresponding second updated coded data to obtain a decoding result includes: For each set of coded data, according to the matching priority between each computing task and each server indicated by the corresponding second updated coded data, each computing task is assigned to the server having the highest matching priority relationship therewith; When all computing tasks of the simulation optimization task queue are allocated and completed, the scheduling server and execution order of each computing task are obtained, and the task completion time of the simulation optimization task queue is calculated.

8. The method according to claim 7, characterized in that The calculation simulation optimization task queue task completion time includes: Get the server corresponding to the last calculation task of each workflow in the simulation optimization task queue; Determine the start time of the last computing task based on the task type of the last computing task and the server corresponding to the last computing task; Determine the task end time of the last calculation task according to the start execution time of the last calculation task and the task prediction time of the last calculation task; Compare the task end time of the last calculation task of each workflow, and take the maximum value as the task completion time of the simulation optimization task queue.

9. The method according to claim 4, characterized in that When the scheduling optimization problem corresponding to the decoding result meets the set conditions, the iteration is determined to be finished, including: When the task completion time of the simulation optimization task queue shown in the current decoding result meets the iteration target requirements of the scheduling optimization problem, each computing task of each workflow meets the constraints, and each available server meets the constraints, it is determined that the scheduling optimization problem corresponding to the current decoding result meets the set conditions and the iteration is determined to be ended.

10. A highly dynamic scheduling system for workflow computing tasks, characterized in that: The system includes: Monitoring module, used to monitor task scheduling events; A first acquisition module is used to acquire a simulation optimization task queue to be scheduled according to the triggering conditions of the triggered task scheduling event when the task scheduling event is triggered, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow; wherein the triggering conditions of the task scheduling event include the addition of a new workflow, a change in task status, a fluctuation in resource load, and a decrease in computing performance; accordingly, acquiring the simulation optimization task queue to be scheduled according to the triggering conditions of the triggered task scheduling event includes: when the triggering condition is the addition of a new workflow, using the new workflow as the simulation optimization task queue to be scheduled; when the triggering condition is a change in task status, a fluctuation in resource load, or a decrease in computing performance, acquiring currently unscheduled computing tasks and determining the currently unscheduled computing tasks as the simulation optimization task queue to be scheduled; The second acquisition module is used to obtain multiple currently available servers and resource characteristics of each server; A construction module is used to construct a scheduling optimization problem based on the task requirements of each workflow and the resource characteristics of each server, and select an optimization algorithm. The scheduling optimization problem includes an iteration goal and constraints. A third acquisition module is used to acquire an encoding method and a decoding method for the optimization algorithm according to the scheduling optimization problem; The determination module is used to use the constraint conditions and iteration targets of the scheduling optimization problem as iteration conditions, and adopt the optimization algorithm, the encoding method and the decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled.

Images