Workflow calculation task high-dynamic scheduling method and system

Abstract

The invention provides a workflow calculation task high-dynamic scheduling method and system. The method comprises the steps that a task scheduling event is monitored; under the condition that the task scheduling event is triggered, a simulation optimization task queue to be scheduled is obtained according to the triggering condition of the triggered task scheduling event, and the simulation optimization task queue comprises a plurality of workflows and the task requirement of each workflow; obtaining a plurality of currently available servers and resource characteristics of each server; according to the task requirement of each workflow and the resource characteristics of each server, a scheduling optimization problem is constructed, an optimization algorithm is selected, and the scheduling optimization problem comprises a constraint condition and an iteration target; according to the scheduling optimization problem, obtaining a coding mode and a decoding mode for an optimization algorithm; and determining a scheduling mode of the simulation optimization task queue to be scheduled by adopting an optimization algorithm, a coding mode and a decoding mode by taking a constraint condition and an iteration target of the scheduling optimization problem as iteration conditions.

Description

Technical Field

[0001] The present application relates to the technical field of highly dynamic scheduling of workflow computing tasks, and in particular to a method and system for highly dynamic scheduling of workflow computing tasks. Background Art

[0002] Scientific workflow scheduling, especially CAE (Computer Aided Engineering) simulation optimization scheduling, faces the following difficulties: 1) The diversity of computing task attributes. Computing tasks include single-node computing and compound-node computing, and each computing task has different levels of difficulty, such as short computing tasks, long computing tasks, and complex resource-consuming tasks. These attributes directly affect the resource requirements and execution time of computing tasks.

[0003] 2) Management of inter-task dependencies: There are complex dependencies between the computational tasks of a composite node and its child nodes, especially when the parent node in the optimization computation job needs to wait for the child node to complete. That is, there is a cyclic convergence problem, which can be expressed as a cyclic dependency in the directed graph (DG), which makes the scheduling problem more complicated. Traditional scheduling methods cannot directly handle DG workflows containing cycles.

[0004] 3) Computing resource prediction and dynamic scheduling. CAE-related simulation tasks cannot accurately estimate the computing resources, including the remaining running time of the computing tasks and the available system resources.

[0005] 4) Directed scheduling based on task attributes. Task attributes are limited, and it is necessary to ensure that tasks can be dispatched to servers that have the resources to solve the task.

[0006] 5) Multi-user multi-task priority. When multiple users submit simulation tasks, how to determine the task priority and queuing strategy is a key issue.

[0007] Therefore, in order to solve the above problems, it is necessary to develop more intelligent and flexible scheduling methods to improve the efficiency and accuracy of CAE simulation optimization. Summary of the invention

[0008] The embodiments of the present application provide a method and system for highly dynamic scheduling of workflow computing tasks.

[0009] According to a first aspect of the present application, a highly dynamic scheduling method for workflow computing tasks is provided, the method comprising: Monitor task scheduling events; When the task scheduling event is triggered, a simulation optimization task queue to be scheduled is obtained according to the triggering condition of the triggered task scheduling event, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow; Get the currently available multiple servers and the resource characteristics of each server; Combine the task requirements of each workflow and the resource characteristics of the server to construct a scheduling optimization problem and select an optimization algorithm; According to the scheduling optimization problem, obtaining an encoding method and a decoding method for the optimization algorithm; The constraint conditions and iteration target of the scheduling optimization problem are used as iteration conditions, and the scheduling method of the simulation optimization task queue to be scheduled is determined by using the optimization algorithm, the encoding method and the decoding method.

[0010] According to an implementation manner of the present application, the triggering conditions of the task scheduling event include the addition of a new workflow, a change in task status, a fluctuation in resource load, and a decrease in computing performance; accordingly, obtaining a simulation optimization task queue to be scheduled according to the triggering conditions of the triggered task scheduling event includes: When the trigger condition is the addition of a new workflow, the new workflow is used as a simulation optimization task queue to be scheduled; When the trigger condition is a task state change, resource load fluctuation, or computing performance degradation, currently unscheduled computing tasks are obtained, and currently unscheduled computing tasks are determined as simulation optimization task queues to be scheduled.

[0011] According to an embodiment of the present application, when the trigger condition is the addition of a new workflow, the step of obtaining the simulation optimization task queue to be scheduled according to the trigger condition of the triggered task scheduling event includes: Acquire task flow data that triggers the task scheduling event, wherein the task flow data includes a plurality of directed acyclic graph workflows and / or a plurality of directed cyclic graph workflows; In the case where the task flow data includes directed cyclic graph workflows, splitting each directed cyclic graph workflow into directed acyclic graph workflows according to the number of task cycles of multiple tasks in each directed cyclic graph workflow; Add all directed acyclic graph workflows to the simulation optimization task queue.

[0012] According to an implementation manner of the present application, the constraint conditions include a first optimization constraint and a second optimization constraint, the task requirements include a task execution order and a task dependency of each computing task in each workflow, and the resource characteristics include resource limitation information of each server; accordingly, According to the task requirements of each workflow and the resource characteristics of each server, a scheduling optimization problem is constructed and an optimization algorithm is selected, including: Determine the last computing task of each workflow according to the task execution order of each computing task shown in the task requirements of each workflow, and minimize the task end time of the last computing task of the currently schedulable workflow as the iteration target; According to the task dependency of each computing task shown in the task requirements of each workflow, determining that the start execution time of each computing task is no earlier than the earliest start time of each computing task as the first optimization constraint, and the earliest start time of each computing task is determined according to the completion time of its predecessor computing task; According to the resource limitation information shown in the resource characteristics of each server, determining that the task resources of all computing tasks received by each server are not greater than the resource limitation information corresponding to each server is the second optimization constraint; According to the task requirements of each workflow and the predicted execution time of each workflow, an optimization algorithm is determined, and the optimization algorithm includes a task scheduling algorithm, a genetic algorithm, a particle swarm optimization, a simulated annealing algorithm, and a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm.

[0013] According to an implementation mode of the present application, the scheduling method includes the scheduling server and execution order of each computing task of each workflow in the simulation optimization task queue to be scheduled; accordingly, Taking the constraint conditions and iteration target of the scheduling optimization problem as iteration conditions, the optimization algorithm, the encoding method and the decoding method are used to determine the scheduling method of the simulation optimization task queue to be scheduled, including: Encoding the simulation optimization task queue based on the optimization algorithm and the encoding method to obtain multiple sets of encoding data, each set of encoding data including the corresponding server and execution order of each computing task in the simulation optimization task queue; Taking the constraints and iteration targets of the scheduling optimization problem as iteration conditions, the decoding method is used to decode each group of encoded data. When the scheduling optimization problem corresponding to the decoding result meets the set conditions, the iteration is determined to be finished, and the current decoding result is determined as the scheduling method of the simulation optimization task queue to be scheduled.

[0014] According to an implementation manner of the present application, the encoding method includes determining a matching priority of each computing task for each server; accordingly, The simulation optimization task queue is encoded based on the optimization algorithm and the encoding method to obtain multiple sets of encoding data, including: The optimization algorithm is used to determine multiple groups of encoding data for the simulation optimization task queue to be scheduled, each group of encoding data includes the matching priority of each computing task of each workflow in the simulation optimization task queue to be scheduled and each server.

[0015] According to an embodiment of the present application, the decoding method includes: Obtain resource status information and task execution status information of each currently available server; Determine the executable tasks in the simulation optimization task queue according to the task requirements of each workflow in the simulation optimization task queue and the task execution status information; Matching each computing task in each set of coded data with the executable computing tasks in the simulation optimization task queue, updating the matching priority of the computing tasks that failed to match to a set value, and obtaining first updated coded data for each set of coded data; According to the resource limitation information and resource status information of each server, determine the server that does not meet the operation requirements in the first updated encoding data corresponding to each set of encoding data, update the matching priority of the server that does not meet the operation requirements to the set value, and obtain the second updated encoding data of each set of encoding data; For each group of encoded data, the computing tasks in the simulation optimization task queue are scheduled according to the corresponding second updated encoded data to obtain a decoding result, which includes the task completion time of the simulation optimization task queue and the execution order and scheduling server of each computing task in the simulation optimization task queue.

[0016] According to an implementation manner of the present application, for each set of coded data, according to the corresponding second updated coded data, the tasks in the simulation optimization task queue are scheduled to obtain a decoding result, including: For each set of coded data, according to the matching priority between each computing task and each server shown by the corresponding second updated coded data, each computing task is assigned to the server having the highest matching priority relationship therewith; When all computing tasks of the simulation optimization task queue are assigned and completed, the scheduling server and execution order of each computing task are obtained, and the task completion time of the simulation optimization task queue is calculated.

[0017] According to an implementation manner of the present application, the calculation simulation optimization task queue task completion time includes: Get the server corresponding to the last computing task of each workflow in the simulation optimization task queue; Determine the start execution time of the last computing task according to the task type of the last computing task and the server corresponding to the last computing task; Determine the task end time of the last computing task according to the start execution time of the last computing task and the task prediction time of the last computing task; The task end time of the last calculation task of each workflow is compared, and the maximum value is taken as the task completion time of the simulation optimization task queue.

[0018] According to an implementation manner of the present application, when the scheduling optimization problem corresponding to the decoding result meets the set conditions, determining that the iteration ends includes: When the task completion time of the simulation optimization task queue shown in the current decoding result meets the iteration target requirements of the scheduling optimization problem, each computing task of each workflow meets the constraints, and each available server meets the constraints, it is determined that the scheduling optimization problem corresponding to the current decoding result meets the set conditions and the iteration ends.

[0019] According to a second aspect of the present application, a highly dynamic scheduling system for workflow computing tasks is provided, the system comprising: Monitoring module, used to monitor task scheduling events; A first acquisition module is used to acquire a simulation optimization task queue to be scheduled according to a triggering condition of the triggered task scheduling event when the task scheduling event is triggered, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow; The second acquisition module is used to acquire multiple currently available servers and resource characteristics of each server; A construction module, used to construct a scheduling optimization problem and select an optimization algorithm according to the task requirements of each workflow and the resource characteristics of each server, wherein the scheduling optimization problem includes an iteration target and constraint conditions; A third acquisition module is used to acquire an encoding method and a decoding method for the optimization algorithm according to the scheduling optimization problem; The determination module is used to use the constraint conditions and iteration targets of the scheduling optimization problem as iteration conditions, and adopt the optimization algorithm, the encoding method and the decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled.

[0020] The method of the embodiment of the present application monitors task scheduling events; when the task scheduling event is triggered, obtains a simulation optimization task queue to be scheduled according to the triggering condition of the triggered task scheduling event, wherein the simulation optimization task queue includes multiple workflows and the task requirements of each workflow; obtains multiple currently available servers and the resource characteristics of each server; constructs a scheduling optimization problem according to the task requirements of each workflow and the resource characteristics of each server, and selects an optimization algorithm; according to the scheduling optimization problem, obtains an encoding method and a decoding method for the optimization algorithm; uses the constraints and iteration targets of the scheduling optimization problem as iteration conditions, and adopts the optimization algorithm, the encoding method and the decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled. A dynamic monitoring mechanism is used to capture task scheduling requirements, comprehensively collect and analyze simulation optimization task queues and server resource information, accurately construct scheduling optimization problems, adaptively select optimization algorithms, and combine specific encoding and decoding methods to achieve intelligent iterative solutions for task scheduling, thereby improving the flexibility and accuracy of scheduling decisions, significantly enhancing scheduling efficiency in complex situations such as large task time spans and unpredictable resource consumption, and ensuring efficient execution of workflows, thereby solving the problem that traditional scheduling methods cannot schedule CAE simulation optimization tasks.

[0021] It should be understood that the teachings of the present application are not required to achieve all of the beneficial effects described above, but specific technical solutions can achieve specific technical effects, and other embodiments of the present application can also achieve beneficial effects not mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein: In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0023] Figure 1 The following is a flowchart of a highly dynamic scheduling method for workflow computing tasks provided by an embodiment of the present application; Figure 2 A schematic diagram of the implementation flow of the scheduling optimization problem and the optimization algorithm determination operation of the highly dynamic scheduling method for workflow computing tasks provided by the embodiment of the present application is shown; Figure 3 A schematic diagram of the implementation process of the task scheduling mode determination operation of the highly dynamic scheduling method for workflow computing tasks provided by an embodiment of the present application is shown; Figure 4A schematic diagram of the implementation flow of the decoding operation of the highly dynamic scheduling method for workflow computing tasks provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0025] Figure 1 A flowchart for implementing a highly dynamic scheduling method for workflow computing tasks provided in an embodiment of the present application is shown.

[0026] refer to Figure 1 , an embodiment of the present application provides a highly dynamic scheduling method for workflow computing tasks, the method comprising: Operation 101 : monitoring task scheduling events.

[0027] In one embodiment of the present application, in order to monitor the task, a task scheduling event is configured for the task in advance, so as to determine whether the task scheduling is required by monitoring whether the task scheduling event is triggered.

[0028] Operation 102 , when a task scheduling event is triggered, a simulation optimization task queue to be scheduled is obtained according to a triggering condition of the triggered task scheduling event, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow.

[0029] In one embodiment of the present application, the triggering conditions of the task scheduling event include the addition of a new workflow, a change in task status, a fluctuation in resource load, and a decrease in computing performance; accordingly, the simulation optimization task queue to be scheduled is obtained according to the triggering conditions of the triggered task scheduling event, including: when the triggering condition is the addition of a new workflow, the new workflow is used as the simulation optimization task queue to be scheduled; when the triggering condition is a change in task status, a fluctuation in resource load, or a decrease in computing performance, currently unscheduled tasks are obtained, and currently unscheduled tasks are determined as the simulation optimization task queue to be scheduled.

[0030] Tasks need to be rescheduled when new workflows are added, task status changes, resource load fluctuations, or performance degradation occurs. Therefore, trigger conditions including new workflow additions, task status changes, resource load fluctuations, and computing performance degradation can be configured for task scheduling events, so that task scheduling events are triggered and task scheduling begins when new workflows are added, task status changes, resource load fluctuations, or performance degradation occurs.

[0031] In the case of a new workflow being added, the new workflow can be directly determined as the simulation optimization task queue to be scheduled. In the case of task status changes, resource load fluctuations, or performance degradation, the task is in the process of scheduling, and it is necessary to obtain unscheduled tasks as the simulation optimization task queue to be scheduled. For the scheduled tasks, in order to improve scheduling efficiency, they will not be rescheduled.

[0032] In one embodiment of the present application, when the trigger condition is the addition of a new workflow, a simulation optimization task queue to be scheduled is obtained according to the trigger condition of the triggered task scheduling event, including: obtaining task flow data that triggers the task scheduling event, the task flow data including multiple directed acyclic graph workflows and / or multiple directed cyclic graph workflows; when the task flow data includes directed cyclic graph workflows, each directed cyclic graph workflow is split into directed acyclic graph workflows according to the number of task cycles of multiple tasks in each directed cyclic graph workflow; and all directed acyclic graph workflows are added to the simulation optimization task queue.

[0033] The workflow of the present application may include directed cyclic graph workflows and directed acyclic graph workflows. When the type of the workflow is a directed cyclic graph workflow, the directed cyclic graph workflow needs to be decomposed into multiple directed acyclic graph workflows. When the workflow is a directed acyclic graph workflow, no processing is performed. When all directed cyclic graph workflows are decomposed, all directed acyclic graph workflows are determined to be simulation optimization task queues to be scheduled.

[0034] Directed cyclic graph workflows can be divided into cases where the number of task cycles is known and cases where the number of task cycles is unknown. In the case where the number of task cycles is known, for each task that needs to be cycled, the task is copied according to its number of cycles to form multiple tasks. When copying, the dependencies of the directed cyclic graph workflow are followed. After the copying is completed, multiple directed acyclic graph workflows are obtained. In the case where the number of task cycles is unknown and the number of cycles depends on real-time data or resource changes, the iterative DAG (Directed Acyclic Graph) generation method is used to split the directed cyclic graph workflow into multiple directed acyclic graph workflows.

[0035] In one embodiment of the present application, multiple workflows in the simulation optimization task queue may be submitted simultaneously by multiple users, and the multiple workflows include multiple computing tasks with dependencies and task execution order, and the computing tasks are CAE simulation optimization tasks.

[0036] Operation 103: Acquire multiple currently available servers and resource characteristics of each server.

[0037] In order to support computing task scheduling, it is usually necessary to configure multiple servers and schedule computing tasks based on multiple servers. However, not all configured servers are available when computing task scheduling is required. Therefore, when scheduling is required, it is necessary to obtain the currently available servers and obtain the resource characteristics of the servers. The resource characteristics of the server may include information such as the server's CPU (Central Processing Unit) and RAM (Random Access Memory).

[0038] In one embodiment of the present application, the task scheduling of the present application supports traditional distributed computing (private deployment) and public cloud, so the multiple servers can be privately deployed servers or public cloud servers. The multiple servers can also include privately deployed servers and public cloud servers at the same time.

[0039] Operation 104 : construct a scheduling optimization problem according to the task requirements of each workflow and the resource characteristics of each server, and select an optimization algorithm. The scheduling optimization problem includes an iteration target and constraints.

[0040] The task requirements usually show the total number of tasks and the task type of each computing task. The resource characteristics of the server can show information such as available resources and resource stability. The complexity of the computing task can be determined based on the task type, and the scale of the computing task can be determined based on the total number of tasks.

[0041] By analyzing the task scale, task complexity, and available resources, we can construct scheduling optimization problems suitable for all workflows, such as maximizing resource utilization, minimizing completion time, etc. The scheduling optimization problem includes constraints and iteration goals.

[0042] Different optimization algorithms can be configured in advance for different task scales, task complexities, and available resources, such as task scheduling algorithms, genetic algorithms, particle swarm optimization, simulated annealing algorithms, or hybrid optimization algorithms. After obtaining the simulation optimization task queue, the corresponding optimization algorithm can be matched by the task scale, task complexity, and available resources of the currently available servers of the simulation optimization task queue.

[0043] In one embodiment of the present application, a scheduling optimization problem is constructed and an optimization algorithm is selected by combining the task requirements of each workflow and the resource characteristics of the server. The task requirements of each workflow and the resource characteristics of each server can also be provided to the user, and the scheduling optimization problem and optimization algorithm provided by the user can be received.

[0044] Operation 105 : acquiring an encoding method and a decoding method for an optimization algorithm according to the scheduling optimization problem.

[0045] The encoding and decoding methods of the optimization algorithm are determined in order to convert the discrete scheduling optimization problem into decision variables that can be recognized by the optimization algorithm through the encoding and decoding methods, so as to couple the optimization algorithm and the scheduling optimization problem.

[0046] The scheduling optimization problem shows the goal of this scheduling task, such as maximizing resource utilization, minimizing completion time, etc. According to different scheduling optimization problems, encoding and decoding methods for different optimization algorithms are configured in advance. After obtaining the scheduling optimization problem, the encoding and decoding methods that match it can be obtained.

[0047] Operation 106 , using the constraints and iteration targets of the scheduling optimization problem as iteration conditions, and using an optimization algorithm, encoding method, and decoding method to determine a scheduling method for the simulation optimization task queue to be scheduled.

[0048] After determining the encoding and decoding methods, the constraints and iteration targets of the scheduling optimization problem are used as the iteration conditions of the optimization algorithm. The simulation optimization task queue is continuously encoded and decoded based on the optimization algorithm and the encoding method. When the scheduling optimization problem is iterated, that is, the algorithm converges, the iteration is determined to be completed. Among them, the decoding result when the iteration is completed is the scheduling method of the simulation optimization task queue to be scheduled. The scheduling method can generally include the execution order of each computing task in the simulation optimization task queue and the scheduling server.

[0049] Therefore, the embodiment of the present application utilizes a dynamic monitoring mechanism to capture task scheduling requirements, comprehensively collects and analyzes simulation optimization task queues and server resource information, accurately constructs scheduling optimization problems, and appropriately selects optimization algorithms, combined with specific encoding and decoding methods, to achieve intelligent iterative solution of task scheduling, improve the flexibility and accuracy of scheduling decisions, significantly enhance the scheduling efficiency in complex situations such as large task time spans and unpredictable resource consumption, ensure efficient execution of workflows, and thus solve the problem that traditional scheduling methods cannot schedule CAE simulation optimization tasks.

[0050] Figure 2 A schematic diagram of the implementation flow of the scheduling optimization problem and optimization algorithm determination operation of the highly dynamic scheduling method for workflow computing tasks provided in an embodiment of the present application is shown.

[0051] refer to Figure 2 In one embodiment of the present application, the constraint condition includes a first optimization constraint and a second optimization constraint, the task requirement includes a task execution order and a task dependency of each computing task in each workflow, and the resource characteristics include resource limitation information of each server; accordingly, the above operation 104 determines the scheduling optimization problem and the optimization algorithm according to the task requirement of each workflow and the resource characteristics of each server, including: Operation 201, determining the last computing task of each workflow according to the task execution order of each computing task shown in the task requirements of each workflow, and minimizing the task end time of the last computing task of the currently schedulable workflow as an iteration target; Operation 202, according to the task dependency of each computing task shown in the task requirements of each workflow, determining that the start execution time of each computing task is not earlier than the earliest start time of each computing task as a first optimization constraint, and the earliest start time of each computing task is determined according to the completion time of its predecessor computing task; Operation 203, determining, according to the resource limitation information shown in the resource characteristics of each server, that the task resources of all computing tasks received by each server are not greater than the resource limitation information corresponding to each server as a second optimization constraint; Operation 204 , determining an optimization algorithm according to the task requirements of each workflow and the predicted execution time of each workflow, wherein the optimization algorithm includes a task scheduling algorithm, a genetic algorithm, a particle swarm optimization algorithm, a simulated annealing algorithm, and a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm.

[0052] The scheduling optimization problem mainly includes iteration objectives and constraints. The iteration objectives are the optimization objectives, and the constraints are the constraints that need to be met, including the first optimization constraints and the second optimization constraints.

[0053] In order to ensure scheduling efficiency, the iteration goal of the scheduling optimization problem is usually configured to minimize the completion time of the simulation optimization task queue. Since there may be execution order issues between workflows due to dependencies between tasks, the shortest completion time of the simulation optimization task queue should be the shortest completion time of the current schedulable workflow. The completion time of the current schedulable workflow can be understood as the task end time of its last calculation task. Therefore, minimizing the task end time of the last calculation task of the current schedulable workflow can be determined as the iteration goal.

[0054] For example, the task end time of the last computation task of the current schedulable workflow can be ,in, represents the last calculation task, w n Indicatesn Currently schedulable workflows, Indicates the last calculation task The server instance, FT Indicates the final completion time of the workflow. Then the iteration target can be ( W )),in, W For a set of workflows, the iteration goal can be understood as minimizing the task completion time of the last computing task in the current schedulable workflow.

[0055] The task execution process usually involves task logic constraints and resource constraints, namely the first optimization constraints and the second optimization constraints. In order to ensure that the tasks are executed in the task execution order shown by the task requirements, the first optimization constraint needs to be configured so that the start execution time of each computing task is not earlier than the earliest start time of each computing task, wherein the earliest start time of each computing task can be determined based on the completion time of its predecessor computing task. The predecessor computing task of each computing task is the task that has a logical relationship with it and is executed before it.

[0056] In order to adapt to the server resources for task scheduling, a second optimization constraint is also configured that the task resources of all computing tasks received by each server are not greater than the resource limitation information corresponding to each server. Instances for different task types are configured on each server, and the task resources of all computing tasks received by each server can be regarded as the task required resources of all instances on each server. The resource limitation information can be the available resources of the server.

[0057] For example, k The task resources required for each task type can be r k , for the i Servers y i All instances of I kij The sum of the required resources can be expressed as , then the second constraint is ,in, I kij Indicates that on each server y i Deployed on j indivual s k An instance of a task type.

[0058] After constructing the scheduling optimization problem, the optimization algorithm is determined according to the task requirements of each workflow and the predicted execution time of each workflow. The optimization algorithms include task scheduling algorithm, genetic algorithm, particle swarm optimization, simulated annealing algorithm, and hybrid optimization algorithm based on heuristic algorithm and evolutionary algorithm.

[0059] The configuration of the optimization algorithm not only needs to consider the task requirements, but also needs to be combined with the predicted execution time of the task. For example, when the task type is relatively simple (such as less than 10 types), the task scale is small (such as less than 500 in total), and the predicted execution time of the task is short (such as the predicted time is less than 1 hour), the corresponding classic scheduling algorithm can be configured, such as the Min-Min task scheduling algorithm and the Max-Min task scheduling algorithm; when the task scale is large, the complexity is high (many task types), and the predicted execution time of the task is long, the optimization algorithm is configured as a genetic algorithm, a particle swarm optimization algorithm, a simulated annealing algorithm, or a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm.

[0060] Therefore, after determining the total number of tasks and the number of task types according to task requirements, and determining the total task predicted execution time according to the predicted execution time of each workflow, the corresponding optimization algorithm can be matched through analysis.

[0061] Figure 3 A schematic diagram of the implementation flow of the task scheduling mode determination operation of the highly dynamic scheduling method for workflow computing tasks provided in an embodiment of the present application is shown.

[0062] refer to Figure 3 In one embodiment of the present application, the scheduling method includes the scheduling server and execution order of each computing task of each workflow in the simulation optimization task queue to be scheduled; accordingly, the above operation 106 uses the constraint conditions and iteration target of the scheduling optimization problem as iteration conditions, and adopts the optimization algorithm, encoding method and decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled, including: Operation 301, encoding the simulation optimization task queue based on the optimization algorithm and the encoding method to obtain multiple sets of encoding data, each set of encoding data includes a server and an execution order corresponding to each computing task in the simulation optimization task queue; Operation 302, using the constraints and iteration targets of the scheduling optimization problem as iteration conditions, decodes each set of encoded data using a decoding method, and when the scheduling optimization problem corresponding to the decoding result meets the set conditions, determines that the iteration ends, and determines the current decoding result as the scheduling method of the simulation optimization task queue to be scheduled.

[0063] Coding is the coupling point between the algorithm and the problem to be optimized, and defines the set of decision variables that the algorithm can understand. For the workflow task scheduling problem, the optimization algorithm generates the representation of the scheduling server and execution order corresponding to each computing task in the simulation optimization task queue through coding. In order to meet the optimization requirements, the simulation optimization task queue needs to be encoded multiple times to generate multiple sets of encoded data containing task scheduling information.

[0064] The encoding is the content that the optimization algorithm can understand, which represents the solution space of the task scheduling problem, while the decoding converts the encoding result into an intuitive and understandable execution order and execution method. During the optimization process, each encoding only needs to be decoded once to generate the corresponding decoding result. The optimization algorithm continuously generates new encodings, decodes each encoding, and evaluates the quality of its corresponding solution, thereby gradually searching for possible optimal solutions.

[0065] In one embodiment of the present application, the condition for the end of the iteration is that the scheduling scheme generated by the decoding result meets the set conditions of the scheduling optimization problem, that is, the task completion time of the simulation optimization task queue corresponding to the current decoding result meets the iteration target requirements, each computing task in each workflow meets the predetermined constraint conditions, and each available server meets the predetermined constraint conditions. When all the above conditions are met, it is determined that the scheduling optimization problem corresponding to the current decoding result meets the set conditions, and the iteration is terminated.

[0066] The iteration goal of the scheduling optimization problem is to minimize the completion time of the last computing task of the current schedulable workflow, that is, to minimize the task completion time of the simulation optimization task queue. Therefore, when decoding each set of encoded data, the decoding result shows that the completion time of the last workflow is minimized, which can be regarded as the task completion time of the simulation optimization task queue meeting the iteration goal requirements. In addition to meeting the requirements of the iteration goal, each computing task and each server must meet the constraints before the scheduling optimization problem corresponding to the decoding result is considered to meet the set conditions and the iteration of the optimization algorithm ends.

[0067] In one embodiment of the present application, the encoding method includes determining the matching priority of each computing task for each server; accordingly, the simulation optimization task queue is encoded based on the optimization algorithm and the decoding method to obtain multiple groups of encoded data, including: using the optimization algorithm to determine multiple groups of encoded data for the simulation optimization task queue to be scheduled, each group of encoded data includes the matching priority of each computing task of each workflow in the simulation optimization task queue to be scheduled and each server.

[0068] The optimization algorithm calculates the matching priority between the workflow task set in the simulation optimization task queue to be scheduled and each server, and generates multiple sets of encoding data. The matching priority is the encoding, which indicates the degree of adaptation between the task and the server. After decoding, the specific combination of the task and the server can be directly obtained.

[0069] For example, the encoding format can be: X ={ p wky | w ∈ W , k ∈ S w , y ∈ Y},in, X To match the priority set, p wky Represents a computational workflow w No. k Tasks for the server Y The matching priority. W Represents the workflow set of the simulation optimization task queue to be scheduled, S w Representing Workflow w The set of all tasks Y Represents the set of available servers. Based on the optimization algorithm combined with the above encoding method, the simulation optimization task queue can be encoded, and the optimal matching priority set X can be calculated through the optimization algorithm.

[0070] Figure 4 A schematic diagram of the implementation flow of the decoding operation of the highly dynamic scheduling method for workflow computing tasks provided in an embodiment of the present application is shown.

[0071] refer to Figure 4 In one embodiment of the present application, the decoding method includes: Operation 401, obtaining resource status information and task execution status information of each currently available server; Operation 402, determining executable computing tasks in the simulation optimization task queue according to the task requirements of each workflow in the simulation optimization task queue and the task execution status information; Operation 403, matching each computing task in each set of coded data with an executable computing task in the simulation optimization task queue, updating the matching priority of the computing task that fails to match to a set value, and obtaining first updated coded data for each set of coded data; Operation 404, determining the servers that do not meet the operation requirements in the first updated encoding data corresponding to each set of encoding data according to the resource restriction information and resource status information of each server, updating the matching priority of the servers that do not meet the operation requirements to a set value, and obtaining the second updated encoding data of each set of encoding data; Operation 405, for each group of encoded data, schedule the computing tasks in the simulation optimization task queue according to its corresponding second updated encoded data to obtain a decoding result, which includes the task completion time of the simulation optimization task queue and the execution order and scheduling server of each computing task in the simulation optimization task queue.

[0072] Each set of coded data includes all servers corresponding to each computing task in the simulation optimization task queue and the matching priority of each server. However, the task scheduling process is ongoing, and accordingly, the server resource occupancy is also constantly changing. Therefore, it is necessary to obtain the current actual task execution status information and resource status information. Among them, the task execution status information is used to show whether the task has been executed, and the resource status information is used to show the resource occupancy of the server.

[0073] After obtaining the task execution status information and resource status information, the currently executable tasks in the simulation optimization task queue, i.e., the tasks whose predecessor tasks have been completed but not yet executed, can be screened out through the task execution status information and the task dependency relationship shown by the task requirements of each workflow. Then, the tasks in each set of coded data are matched with the currently executable tasks. If no match is found, it means that the unmatched tasks do not need to participate in this iteration. The matching priorities corresponding to the unmatched tasks in each set of coded data are configured as set values, and the first updated coded data corresponding to each set of coded data is obtained.

[0074] After filtering out the tasks that do not meet the requirements, it is also necessary to filter out the servers that do not meet the requirements, that is, for each computing task in the first updated coding data corresponding to each group of coding data, determine whether the resources of the server with a matching priority relationship with each computing task can support the execution of the corresponding task based on the resource limitation information and resource status information of each server. If it cannot be supported, the matching priority between the task and the server is set to a set value to obtain the second updated coding data.

[0075] In one embodiment of the present application, the set value may be infinite.

[0076] After each set of coded data is updated, each computing task in the simulation optimization task queue is dynamically allocated using the second updated coded data, so that scheduling can be achieved and a decoding result can be obtained. The decoding result can be regarded as a scheduling method, including a scheduling server and an execution order for each computing task in the simulation optimization task queue.

[0077] In one embodiment of the present application, for each group of encoded data, tasks in the simulation optimization task queue are scheduled according to the corresponding second updated encoded data to obtain a decoding result, including: for each group of encoded data, according to the matching priority between each computing task and each server shown by the corresponding second updated encoded data, each computing task is assigned to the server with the highest matching priority (lowest numerical value) relationship with it; when all tasks in the simulation optimization task queue are assigned, the scheduling server and execution order of each computing task are obtained, and the task completion time of the simulation optimization task queue is calculated.

[0078] After determining the second updated coding data, the computing task, workflow and server combination with the highest matching priority (i.e., the earliest execution start or the lowest resource consumption) can be found among all currently executable computing tasks and server combinations according to the second updated coding data, and the computing tasks can be allocated based on the combination. Thereafter, all computing tasks can be allocated by continuously repeating the above process of determining the computing task, workflow and server combination with the highest matching priority and allocating them based on the combination, thereby completing the allocation of all computing tasks.

[0079] After all computing tasks are assigned, the decoding result for each set of encoded data can be obtained by obtaining the server to which each computing task is assigned and the execution order.

[0080] After obtaining the decoding results, it is also necessary to calculate the task completion time of the simulation optimization task queue.

[0081] In one embodiment of the present application, the task completion time of the simulation optimization task queue is calculated, including: obtaining the server corresponding to the last computing task of each workflow in the simulation optimization task queue; determining the start execution time of the last computing task according to the task type of the last computing task and the server corresponding to the last computing task; determining the task end time of the last computing task according to the start execution time of the last computing task and the task prediction time of the last computing task; comparing the task end time of the last computing task of each workflow, and taking the maximum value as the task completion time of the simulation optimization task queue.

[0082] The multiple servers currently available can be viewed as a collection of server resources: Y ={ y 1,……, yh}, h is the total number of available servers. The resource characteristics of each server include resource limitation information. The resource limitation information can be expressed as y i =[ cpu i , ram i ], cpu i Indicates i CPU information of each server, ram i Indicates i RAM information of each server.

[0083] Tasks usually have m task types, the set of task types can be S ={ S 1,……, S m Each computing task type can be deployed as an instance, and each computing task type has different task requirements. r =[ r 1,……, r m ] 2xm , then each type of task S m The requirements for CPU and RAM can be expressed as r m =[ cpu m , ram m ] T Each server can deploy one instance or multiple instances, that is, a server can support one task type or multiple task types. I kij Indicates that the server i Deployed on j indivual S k An instance of a task type.

[0084] The tasks are shown in the task requirements for each workflow. s t The dependency relationship between them, that is, the task dependency relationship of each computing task, the dependency set between tasks can be represented by E Indicates that the workflow can be w Indicates that multiple workflows w Form a workflow collection: W ={ w 1,w 2…… w n}. ,in, e pk Indicates dependencies, such as e pk =0, it means the task and There is no data dependency between e pk ≠0, then its value represents the task Towards The amount of data transferred.

[0085] Therefore, each computation task The completion time of each computation task can be considered as In the example I kij The completion time (Finish Time, FT) on FT ( , I kij )= ST ( , I kij )+ ET ( ),in, ST ( , I kij ) refers to the task In the example I kij The time when the task starts execution, that is, At the start execution time of the corresponding server, ET ( ) refers to the task Predict the time it will take to complete the task The task prediction time can be estimated based on the task type and historical data.

[0086] Among them, the task At the start time of the corresponding server ST( ，I kij ) Need to satisfy constraints ST( ， I kij )≥EST( ，I kij) , EST( ，I kij ) It's a task In the example I kij The earliest start time (EST) on the task The earliest completion time on the server, determined by the instance I kij The available time is determined by EST( ，I kij )=max{FT( ，I pi'j' )} , ∈ pred(I pi'j' ) , i' It is to carry out the task Servers, j' yes Belongs to j' instances.

[0087] By obtaining the task completion time of the last calculation task through the above-mentioned method of determining the task completion time, the task completion time of each workflow can be obtained. The maximum value of the task completion time of each workflow can be taken to obtain the task completion time of the simulation optimization task queue.

[0088] For example, the decoding process of the embodiment of the present application may include: S1, obtain the server resource status and simulation optimization task queue.

[0089] Get the resource status information (resource status) of available servers and the simulation optimization task queue to be scheduled.

[0090] S2, filter out the currently computable task set of each workflow .

[0091] According to the task execution order and task dependency shown in the task requirements of each workflow, determine all tasks that have been completed by the predecessor task as currently computable tasks ,in, w Represents a workflow.

[0092] S3, order ,right .

[0093] in, Indicates initialization of a set of encoded data As the encoded data required for the current decoding, Represents a workflow set consisting of multiple workflows in the simulation optimization task queue to be scheduled Any workflow in w , Indicates that for tasks that are not currently computable Tasks in k , Indicates that it will be matched with priority Set to infinity (inf). The purpose is to prevent these tasks or server combinations from being considered in the current iteration.

[0094] S4, yes , Y , determine the server Whether the subtask is satisfied If the operation requirements are not met, .

[0095] For each workflow w The current computable tasks in Tasks in k , check the server y Is the task met? k The running requirements (such as CPU, memory, software dependencies, etc.) of the task are met. If they are not met, the task k On the server y The match priority on is set to infinity (inf).

[0096] S5, .

[0097] Among all currently computable tasks and server combinations, find the task with the lowest priority value (i.e. the earliest start or the best resource consumption) k' , Workflow w' and server y' Combination of .

[0098] S6, the workflow No. Tasks are assigned to the server , calculate the current task The earliest start time and end time of the simulation optimization task queue are updated.

[0099] S7, check whether all tasks have been assigned, if so, go to step S8, otherwise go to step S2, and execute S2-S6 in a loop until all tasks are assigned.

[0100] S8, calculates the time required for all tasks to be completed.

[0101] In one embodiment of the present application, the optimization algorithm is a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm, with the constraints and iteration targets of the scheduling optimization problem as iteration conditions, and each group of encoded data is decoded in a decoding manner. When the scheduling optimization problem corresponding to the decoding result meets the set conditions, the iteration is determined to be finished, and the current decoding result is determined as the scheduling method of the simulation optimization task queue to be scheduled, including: 1) Initialization.

[0102] Set the parameters of the evolutionary algorithm, such as the population size Np , crossover probability C , mutation factor F wait.

[0103] Generate an initial population, each individual in the population represents a candidate solution x , that is, randomly generate multiple sets of coded data p wky , .

[0104] 2) Evaluate the population.

[0105] Calculate the fitness value of each individual in the initial population.

[0106] 3) Iterative evolution.

[0107] Perform mutation operation: For each individual in the population, randomly select three different individuals from the population. Generate mutation vector: ,in, x r1 , x r2 , x r3 are three randomly selected individuals, F It is a factor of variation.

[0108] Perform crossover operation: for each individual x i and its mutation vector v i Perform crossover operation to generate experimental individuals u i , if the random number is less than the crossover probability or j is a randomly selected dimension, then u i (j)=vi (j) ,otherwise u i (j)=x i (j) .

[0109] Perform selection operation: Calculate the experimental individuals u i The fitness value of u i The fitness value is better than x i , then use u i replace x i , otherwise keep x i .

[0110] 4) Neighborhood search (performed after each fixed interval iteration step).

[0111] Perform a field search on some or all individuals in the current population. The field search steps include: 1. Select an individual x i As the initial solution; 2. x i Generate several neighborhood solutions in the neighborhood of x i '(usually generated by random perturbations in each dimension); 3. Calculate the fitness values ​​of these neighborhood solutions; 4. If a neighborhood solution is found x i ' has a better fitness value than x i , then use x i 'replace x i .

[0112] 5) Termination conditions.

[0113] Determine whether the termination conditions are met. If the maximum number of iterations is reached or the fitness value is no longer significantly improved, the above Figure 4 The decoding method described in the description calculates the task completion time of the simulation optimization task queue.

[0114] If the task completion time of the simulation optimization task queue reaches the shortest, the termination condition is determined to be met, the algorithm ends, and the optimal solution is output. x , if the termination condition is not met, return to step 3) to continue iterating.

[0115] 6) Output the final scheduling method.

[0116] The final scheduling method is the optimal solution x The corresponding decoding result. According to the optimal solution x and the above Figure 4 The decoding method determines the task completion time of the simulation optimization task queue, the execution order of each computing task in the simulation optimization task queue and the scheduling server.

[0117] Therefore, the present application combines the global search capability of the evolutionary algorithm and the local optimization capability of the domain search to effectively explore the solution space and improve the scheduling efficiency.

[0118] Based on the above method, an embodiment of the present application also provides a highly dynamic scheduling system for workflow computing tasks, the system comprising: a monitoring module for monitoring task scheduling events; a first acquisition module for, when a task scheduling event is triggered, acquiring a simulation optimization task queue to be scheduled according to the triggering conditions of the triggered task scheduling event, the simulation optimization task queue comprising multiple workflows and the task requirements of each workflow; a second acquisition module for acquiring multiple currently available servers and the resource characteristics of each server; a construction module for constructing a scheduling optimization problem and selecting an optimization algorithm according to the task requirements of each workflow and the resource characteristics of each server, the scheduling optimization problem comprising iteration objectives and constraints; a third acquisition module for acquiring an encoding method and a decoding method for the optimization algorithm according to the scheduling optimization problem; a determination module for determining the scheduling method of the simulation optimization task queue to be scheduled using the constraints and iteration objectives of the scheduling optimization problem as iteration conditions, and adopting the optimization algorithm, encoding method and decoding method.

[0119] It should be noted that the description of the highly dynamic scheduling system for workflow computing tasks in the embodiment of the present application is similar to the description of the above-mentioned method embodiment, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figures 1 to 4 The present invention can be understood by referring to the description of any one of the accompanying drawings.

[0120] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0121] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A highly dynamic scheduling method for workflow computing tasks, characterized in that: The method comprises: Monitor task scheduling events; When the task scheduling event is triggered, a simulation optimization task queue to be scheduled is obtained according to the triggering condition of the triggered task scheduling event, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow; Get the currently available multiple servers and the resource characteristics of each server; According to the task requirements of each workflow and the resource characteristics of each server, a scheduling optimization problem is constructed and an optimization algorithm is selected, wherein the scheduling optimization problem includes an iteration target and constraints; According to the scheduling optimization problem, obtaining an encoding method and a decoding method for the optimization algorithm; The constraint conditions and iteration target of the scheduling optimization problem are used as iteration conditions, and the scheduling method of the simulation optimization task queue to be scheduled is determined by using the optimization algorithm, the encoding method and the decoding method.

2. The method according to claim 1, characterized in that The triggering conditions of the task scheduling event include the addition of a new workflow, a change in task status, a fluctuation in resource load, and a decrease in computing performance; accordingly, The step of obtaining the simulation optimization task queue to be scheduled according to the triggering condition of the triggered task scheduling event includes: When the trigger condition is the addition of a new workflow, the new workflow is used as a simulation optimization task queue to be scheduled; When the trigger condition is a task state change, resource load fluctuation, or computing performance degradation, currently unscheduled computing tasks are obtained, and currently unscheduled computing tasks are determined as simulation optimization task queues to be scheduled.

3. The method according to claim 2, characterized in that In the case where the trigger condition is the addition of a new workflow, the step of obtaining the simulation optimization task queue to be scheduled according to the trigger condition of the triggered task scheduling event includes: Acquire task flow data that triggers the task scheduling event, wherein the task flow data includes a plurality of directed acyclic graph workflows and / or a plurality of directed cyclic graph workflows; In the case where the task flow data includes directed cyclic graph workflows, splitting each directed cyclic graph workflow into directed acyclic graph workflows according to the number of task cycles of multiple tasks in each directed cyclic graph workflow; Add all directed acyclic graph workflows to the simulation optimization task queue.

4. The method according to claim 1, wherein the constraint condition comprises a first optimization constraint and a second optimization constraint, the task requirement comprises a task execution order and a task dependency of each computing task in each workflow, and the resource characteristics comprises resource limitation information of each server; accordingly, According to the task requirements of each workflow and the resource characteristics of each server, a scheduling optimization problem is constructed and an optimization algorithm is selected, including: Determine the last computing task of each workflow according to the task execution order of each computing task shown in the task requirements of each workflow, and minimize the task end time of the last computing task of the currently schedulable workflow as the iteration target; According to the task dependency of each computing task shown in the task requirements of each workflow, determining that the start execution time of each computing task is no earlier than the earliest start time of each computing task as the first optimization constraint, and the earliest start time of each computing task is determined according to the completion time of its predecessor computing task; According to the resource limitation information shown in the resource characteristics of each server, determining that the task resources of all computing tasks received by each server are not greater than the resource limitation information corresponding to each server is the second optimization constraint; According to the task requirements of each workflow and the predicted execution time of each workflow, an optimization algorithm is determined, and the optimization algorithm includes a task scheduling algorithm, a genetic algorithm, a particle swarm optimization, a simulated annealing algorithm, and a hybrid optimization algorithm based on a heuristic algorithm and an evolutionary algorithm.

5. The method according to claim 1, characterized in that The scheduling method includes the scheduling server and execution order of each computing task of each workflow in the simulation optimization task queue to be scheduled; accordingly, Taking the constraint conditions and iteration target of the scheduling optimization problem as iteration conditions, the optimization algorithm, the encoding method and the decoding method are used to determine the scheduling method of the simulation optimization task queue to be scheduled, including: Encoding the simulation optimization task queue based on the optimization algorithm and the encoding method to obtain multiple sets of encoding data, each set of encoding data including the server and execution order corresponding to each computing task in the simulation optimization task queue; Taking the constraints and iteration targets of the scheduling optimization problem as iteration conditions, the decoding method is used to decode each group of encoded data. When the scheduling optimization problem corresponding to the decoding result meets the set conditions, the iteration is determined to be finished, and the current decoding result is determined as the scheduling method of the simulation optimization task queue to be scheduled.

6. The method according to claim 5, characterized in that The encoding method includes determining the matching priority of each computing task for each server; accordingly, The simulation optimization task queue is encoded based on the optimization algorithm and the encoding method to obtain multiple sets of encoding data, including: The optimization algorithm is used to determine multiple groups of encoding data for the simulation optimization task queue to be scheduled, each group of encoding data includes the matching priority of each computing task of each workflow in the simulation optimization task queue to be scheduled and each server.

7. The method according to claim 6, characterized in that The decoding method includes: Obtain resource status information and task execution status information of each currently available server; Determine the executable computing tasks in the simulation optimization task queue according to the task requirements of each workflow in the simulation optimization task queue and the task execution status information; Matching each computing task in each set of coded data with the executable computing tasks in the simulation optimization task queue, updating the matching priority of the computing tasks that failed to match to a set value, and obtaining first updated coded data for each set of coded data; According to the resource limitation information and resource status information of each server, determine the server that does not meet the operation requirements in the first updated encoding data corresponding to each set of encoding data, update the matching priority of the server that does not meet the operation requirements to the set value, and obtain the second updated encoding data of each set of encoding data; For each group of encoded data, the computing tasks in the simulation optimization task queue are scheduled according to the corresponding second updated encoded data to obtain a decoding result, which includes the task completion time of the simulation optimization task queue and the execution order and scheduling server of each computing task in the simulation optimization task queue.

8. The method according to claim 7, characterized in that For each set of coded data, according to the corresponding second updated coded data, the computing tasks in the simulation optimization task queue are scheduled to obtain a decoding result, including: For each set of coded data, according to the matching priority between each computing task and each server shown by the corresponding second updated coded data, each computing task is assigned to the server having the highest matching priority relationship therewith; When all computing tasks of the simulation optimization task queue are assigned and completed, the scheduling server and execution order of each computing task are obtained, and the task completion time of the simulation optimization task queue is calculated.

9. The method according to claim 8, characterized in that The calculation simulation optimization task queue task completion time includes: Get the server corresponding to the last computing task of each workflow in the simulation optimization task queue; Determine the start execution time of the last computing task according to the task type of the last computing task and the server corresponding to the last computing task; Determine the task end time of the last computing task according to the start execution time of the last computing task and the task prediction time of the last computing task; The task end time of the last calculation task of each workflow is compared, and the maximum value is taken as the task completion time of the simulation optimization task queue.

10. The method according to claim 5, characterized in that When the scheduling optimization problem corresponding to the decoding result meets the set conditions, determining the end of the iteration includes: When the task completion time of the simulation optimization task queue shown in the current decoding result meets the iteration target requirements of the scheduling optimization problem, each computing task of each workflow meets the constraints, and each available server meets the constraints, it is determined that the scheduling optimization problem corresponding to the current decoding result meets the set conditions and the iteration ends.

11. A highly dynamic scheduling system for workflow computing tasks, characterized in that: The system includes: Monitoring module, used to monitor task scheduling events; A first acquisition module is used to acquire a simulation optimization task queue to be scheduled according to a triggering condition of the triggered task scheduling event when the task scheduling event is triggered, wherein the simulation optimization task queue includes multiple workflows and task requirements of each workflow; The second acquisition module is used to acquire multiple currently available servers and resource characteristics of each server; A construction module, used to construct a scheduling optimization problem and select an optimization algorithm according to the task requirements of each workflow and the resource characteristics of each server, wherein the scheduling optimization problem includes an iteration target and constraint conditions; A third acquisition module is used to acquire an encoding method and a decoding method for the optimization algorithm according to the scheduling optimization problem; The determination module is used to use the constraint conditions and iteration targets of the scheduling optimization problem as iteration conditions, and adopt the optimization algorithm, the encoding method and the decoding method to determine the scheduling method of the simulation optimization task queue to be scheduled.

Images