Distributed workflow load parallel scheduling method and device based on Monte Carlo

Abstract

The invention relates to the technical field of information, in particular to a distributed workflow load parallel scheduling method and device based on Monte Carlo. The method comprises the following steps: sampling execution time by using a Monte Carlo method; multiple scheduling schemes are generated based on static scheduling CPOP, after a critical path is obtained by using a CPOP algorithm, the search time is shortened by using a Monte Carlo search tree, and the scheduling schemes with long time are screened out; and sampling the execution time, calculating the minimum scheduling time obtained according to different scheduling schemes, and selecting the scheduling scheme with the minimum average value of the minimum scheduling time. The invention aims to break through the limitation of the traditional static scheduling method in dealing with the uncertainty of the task execution time, and more accurately grasp the randomness characteristic of the task execution time by means of analogue simulation and sampling analysis so as to realize a better scheduling strategy and overall performance optimization in a decision making process.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a Monte Carlo-based distributed workflow load parallel scheduling method and device. Background Art

[0002] The rapid development of grid computing and cloud computing technologies in recent years has provided strong support for large-scale scientific computing applications. These technology platforms aggregate vast amounts of high-performance distributed resources via the internet and integrate them into resource pools, enabling applications to flexibly access computing power, storage space, and diverse software services based on actual needs. Distributed system architectures such as grid workflows and cloud computing workflows, currently under in-depth exploration and development, not only enable the efficient construction and implementation of large-scale DAG-like computing tasks but also fully leverage the vast computing resources provided by grid and cloud computing. These systems have successfully achieved highly automated and efficient operation of such DAG tasks in numerous scientific computing fields. However, skillfully scheduling the subtasks in a DAG across multiple computing resources to minimize overall task completion time or achieve other optimization goals is an extremely complex and challenging task scheduling optimization problem. In this process, not only must the processing power differences between different computers be balanced, but the potential for workflow deadlocks due to resource squeezes must also be considered. Therefore, how to minimize deadlocks while maximizing workflow efficiency is a worthy research topic.

[0003] Scheduling algorithms are primarily categorized into two main types: dynamic and static. In dynamic scheduling, the specific computational and communication costs of each task, as well as the inter-task relationships, are often unknown at the outset of scheduling. Therefore, these algorithms offer strong real-time adaptability. Whenever a new task arrives, a dynamic scheduling algorithm can rapidly generate a real-time scheduling decision based on the specific parameters at that time to accommodate changing task demands. In contrast, static scheduling algorithms already know the detailed computational and inter-task communication costs of each task before execution. This means that during the scheduling process, the algorithm can directly generate a complete scheduling result based on pre-set parameters. Representative categories of static scheduling algorithms include list scheduling, task replication-based scheduling, task clustering-based scheduling, and random search algorithms that leverage genetic and evolutionary principles. The core concept of list scheduling is to construct an ordered scheduling list by calculating the priority of each task. During scheduling, the system assigns tasks to appropriate processors based on their priority within the list. Notably, the Predicted Earliest Finish Time (PEFT) algorithm proposed by Arabnejad constructs an optimistic cost table (OCT) to calculate the minimum sum of the computational and communication costs of all subtasks on different processors and selects the optimal processor based on this value. There are also heterogeneous earliest finish time algorithms (HEFT) and critical path on processor algorithms (CPOP).

[0004] In summary, many currently used heuristic scheduling strategies tend to treat task execution times as constant values, often determining specific execution times at the outset of scheduling. Even in complex situations where task execution times are expressed as probability distributions, these static scheduling methods can perform computational planning by extracting a statistical parameter of this distribution (such as the mean). However, in many practical scenarios, this approach fails to achieve optimal scheduling results, especially considering the potential deadlock caused by resource competition in multi-workflow scheduling environments. Summary of the Invention

[0005] The embodiments of the present invention provide a Monte Carlo-based distributed workflow load parallel scheduling method and device, so as to at least solve the technical problem of poor effect of existing workflow load parallel scheduling.

[0006] According to an embodiment of the present invention, a Monte Carlo-based distributed workflow load parallel scheduling method is provided, comprising the following steps:

[0007] S101: Using the Monte Carlo method to sample the execution time;

[0008] S102: generating multiple scheduling schemes based on static scheduling CPOP, using CPOP algorithm to obtain the critical path, and then using Monte Carlo search tree to reduce search time, and screening out scheduling schemes with long execution time;

[0009] S103: sampling the execution time, calculating the minimum scheduling time obtained according to different scheduling schemes, and selecting the scheduling scheme with the minimum mean scheduling time.

[0010] Further, using CPOP algorithm to obtain the critical path and then using Monte Carlo search tree to reduce search time comprises:

[0011] Using Monte Carlo simulation method to predict the task execution time set and generating the optimal task execution order.

[0012] Further, in step S102, according to the optimal task execution order, the scheduling scheme with the shortest time is selected by screening out the scheduling scheme with long execution time.

[0013] Further, the method further comprises:

[0014] In the case of multiple task flows and data exchange between tasks, the Monte Carlo search tree scheduling strategy is used to prevent deadlock.

[0015] Further, the Monte Carlo search tree scheduling strategy for preventing deadlock comprises:

[0016] Whenever there is a new DAG flow or the current node of the DAG flow ends the task, before executing the new task node, the capacity of the currently available resources is obtained to build a Monte Carlo tree for n times of simulation, and once the simulation times reach the corresponding value, all nodes are traversed.

[0017] Further, the four steps of Monte Carlo search tree include selection, expansion, simulation and back propagation.

[0018] Further, in the selection phase, a node most urgently needing to be expanded is selected from the root node of the situation to be decided, and a point not selected is selected as much as possible.

[0019] Further, in the expansion phase, a node and an action not yet expanded of the node are found, other workflow scheduling sequences previously calculated are scheduled according to the action, and the current required resource state is obtained.

[0020] Further, in the current required resource state, if the current resource cannot support the resource scheduling to be completed, it is recorded as 0 in the backtracking phase, and if all nodes can complete the task after scheduling, the other scheduling sequences previously obtained by the workflow are rescheduled.

[0021] According to another embodiment of the present application, a parallel scheduling device based on Monte Carlo distributed workflow load is provided, comprising:

[0022] a sampling unit for sampling execution time by using a Monte Carlo method;

[0023] a scheduling scheme acquisition unit for generating a plurality of scheduling schemes based on static scheduling CPOP, using a CPOP algorithm to obtain a critical path, and then using a Monte Carlo search tree to reduce search time and screen out scheduling schemes with long execution time;

[0024] a scheduling scheme selection unit for sampling execution time, calculating minimum scheduling time obtained according to different scheduling schemes, and selecting a scheduling scheme with minimum mean minimum scheduling time.

[0025] A storage medium, which stores a program file capable of implementing any one of the above-mentioned parallel scheduling methods based on Monte Carlo distributed workflow load.

[0026] A processor for running a program, wherein the program performs any one of the above-mentioned parallel scheduling methods based on Monte Carlo distributed workflow load when running.

[0027] The parallel scheduling method and device based on Monte Carlo distributed workflow load in the embodiments of the present application propose a DAG (directed acyclic graph) preprocessing scheme relying on Monte Carlo simulation technology, which aims to fully utilize the probability distribution attribute of task execution time, rather than being limited to single evaluation estimation. This method aims to break through the limitations of traditional static scheduling methods in dealing with task execution time uncertainty, and more accurately grasp the randomness characteristics of task execution time through simulation and sampling analysis means, so as to realize a more optimal scheduling strategy and overall performance optimization in the decision-making process. BRIEF DESCRIPTION OF DRAWINGS

[0028] The accompanying drawings, which are included to provide a further understanding of the present application and constitute a part of this application, illustrate certain illustrative embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0029] Figure 1 a flowchart of the parallel scheduling method based on Monte Carlo distributed workflow load of the present application;

[0030] Figure 2 a task scheduling flowchart in the present application;

[0031] Figure 3 an actual scheduling algorithm diagram in the present application;

[0032] Figure 4A simple block diagram of the scheduling algorithm in the present application;

[0033] Figure 5 An algorithm diagram for solving the deadlock in the present application;

[0034] Figure 6 A code diagram of the Monte Carlo tree search in the present application;

[0035] Figure 7 A module diagram of the distributed workflow load parallel scheduling device based on Monte Carlo in the present application. DETAILED DESCRIPTION

[0036] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiment of the present application will be described clearly and completely in combination with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, not all. Based on the embodiment in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0038] Embodiment 1

[0039] According to an embodiment of the present application, a Monte Carlo-based distributed workflow load parallel scheduling method is provided, referring to Figure 1 , comprising the following steps:

[0040] S101: sampling the execution time by using the Monte Carlo method;

[0041] S102: generating a plurality of scheduling schemes based on static scheduling CPOP, using the CPOP algorithm to obtain the critical path, and then using the Monte Carlo search tree to reduce the search time and screen out the scheduling schemes with long execution time;

[0042] S103: Sampling the execution time, calculating the minimum scheduling time according to different scheduling schemes, and selecting the scheduling scheme with the smallest mean minimum scheduling time.

[0043] The Monte Carlo-based distributed workflow workload parallel scheduling method in this embodiment of the present invention proposes a DAG (directed acyclic graph) preprocessing scheme based on Monte Carlo simulation technology. This scheme aims to fully utilize the probabilistic distribution properties of task execution time, rather than being limited to a single estimate. This method aims to overcome the limitations of traditional static scheduling methods in dealing with the uncertainty of task execution time. Through simulation and sampling analysis, it can more accurately grasp the random characteristics of task execution time, thereby achieving better scheduling strategies and overall performance optimization during the decision-making process.

[0044] The present invention proposes a DAG (directed acyclic graph) preprocessing scheme based on Monte Carlo simulation technology, which aims to make full use of the probabilistic distribution properties of task execution time, rather than being limited to the estimation of a single valuation. This method aims to break through the limitations of traditional static scheduling methods in dealing with the uncertainty of task execution time, and through simulation and sampling analysis, more accurately grasp the random characteristics of task execution time, in order to achieve better scheduling strategies and overall performance optimization in the decision-making process. Secondly, in order to avoid the deadlock problem caused by resource runs in multi-workflow scenarios, the present invention designs an algorithm to solve deadlock.

[0045] The technical solution of the present invention is described in detail as follows:

[0046] Monte Carlo Tree Search (MCTS), an effective heuristic search technique that combines the universality of random sampling with the precision of tree-structured search, has achieved breakthroughs in the field of game AI, particularly in various gaming environments such as Go and Texas Hold'em. Its operating mechanism revolves around five key steps:

[0047] Selection: Starting from the root node, a recursive search selects child nodes according to a strategy (such as the Unified Decision Tree (UCD) formula). This process continues down the tree path until an unexpanded node is encountered or the preset search depth is reached. The goal of the selection strategy is to find the most promising child nodes among the explored nodes while maintaining a certain degree of exploratory power. For example, the Unified Decision Tree (UCT) formula combines the cumulative reward and the number of visits to a node to balance exploration and exploitation.

[0048] Expansion: If the selected node is not fully expanded (i.e., there are unexplored actions), select one of the unexplored actions to expand, generating a new child node. The goal of expansion is to enlarge the search space in order to discover new possibilities. For example, in a board game, expansion can represent trying different move positions.

[0049] Simulation: Starting from the newly generated child node, perform a complete random simulation (also known as a "rollout"). During this process, no search strategy is used, and a simple random strategy or default strategy is employed for decision-making until an end condition is reached (e.g., game over or reaching the maximum number of steps). The goal of simulation is to evaluate the potential value of the current node in order to make better decisions in subsequent searches. For example, in a board game, simulation can represent playing a few steps of random moves after a random move to observe the final win or lose situation.

[0050] Back propagation: Propagate the results of simulation (usually the final reward) back to the parent node, updating the statistical information of each node, such as visit count and cumulative reward. The goal of back propagation is to collect more information in order to make more accurate decisions in future searches. For example, in a board game, back propagation can represent feeding the win or lose result of simulation back to the parent node to update its cumulative reward and visit count.

[0051] Repeat: Go back to the selection phase and continue the next round of selection, expansion, simulation, and back propagation until the preset search number is reached or other stopping conditions are met. The goal of repetition is to improve the quality of search through multiple iterations to find a solution closer to the optimal solution. For example, in a board game, repetition can represent performing multiple rounds of search to find the best move position.

[0052] In simple terms, it is to use the Monte Carlo method to sample the execution time, and based on the static scheduling CPOP to generate multiple scheduling schemes. Using the CPOP algorithm is to get the critical path and then use the Monte Carlo search tree to reduce the search time, and filter out the scheduling schemes with longer execution time. From this group of schemes, the best one is selected. This also samples the execution time, calculates the minimum scheduling time obtained according to different scheduling schemes, and selects the one with the smallest average minimum scheduling time.

[0053] Meanwhile, in an ideal case, if there are a large number of available resources and the DAG exists data dependency, the task scheduler can maximize the parallelism. However, this ideal case is not always feasible in reality. When resources are constrained, there may be a deadlock problem according to the model, because when waiting for the completion of a subtask, a task is likely to hold the allocated resources, and the subtask may be blocked due to insufficient available resources for execution. For example, in Figure 2 , (the bidirectional arrow indicates that there is a dependency between the data), if sufficient resources R1 are allocated to tasks A, B, E, E1, D, D2, C, H, H1 along the corresponding path for execution, only 10 R1 units can be released after tasks E1, D2, H, H1 are completed. If there are no more additional R1 resources available, then such a number of resources is insufficient to allow the program to continue, thereby causing a deadlock, regardless of what happens to resource R2.

[0054] For the above case, the present application proposes a design to solve the deadlock, as shown in Figures 3-6 , whenever there is a new DAG flow or the current node of the DAG flow ends the task, before executing the new task node, the capacity of the currently available resources is obtained to establish a Monte Carlo tree for n times of simulation, and once the simulation times reach the corresponding value, if all nodes are successfully executed, it means that the current task can be executed to completion, and therefore the premise of entering the execution queue for the task is that the task can be executed to completion. Therefore, the situation of insufficient resources to support the next task due to the occupation of resources by each task does not occur, thereby avoiding the deadlock phenomenon caused by resource preemption. In this part, the four steps of the Monte Carlo search tree are still selection, expansion, simulation and back propagation. In the selection phase, a node that needs to be expanded most urgently is selected from the root node, that is, the situation where a decision needs to be made, but the present application will try to select a point that has not been selected (also the point with the largest UCT value). The expansion phase is that when the selection phase ends, the present application finds a node and an action that has not been expanded, and the action schedules the scheduling sequence previously calculated by other workflows. At this time, the present application obtains the current resource state, and if the current resources of the present application cannot support the completion of resource scheduling, the present application will be recorded as 0 in the backtracking phase. If all nodes can complete the task after scheduling, the present application will reschedule the other scheduling sequences previously obtained by the workflow, and the existence of UCT will make the present application preferentially schedule the node with fewer repeated nodes, hoping to end the deadlock detection as soon as possible.

[0055] The key points and points to be protected of the present application are:

[0056] 1. In order to achieve the goal of minimizing the total time of workflow scheduling, the present application designs a two-stage implemented scheduling algorithm. First, the Monte Carlo simulation method is used to predict the task execution time set, so as to generate the optimal task execution order.

[0057] 2. Secondly, according to the obtained sorting result, the results with relatively long time are screened out, and the Monte Carlo search tree technology is combined to skillfully perform time sampling prediction on the scheduling sequence in the queue.

[0058] 3. In the case of multiple task flows and data exchange between tasks, a scheduling strategy is proposed to effectively prevent the occurrence of deadlock.

[0059] Compared with the prior art, the present application has the following advantages:

[0060] The present application proposes a DAG (Directed Acyclic Graph) scheduling technology based on the Monte Carlo method, which can better utilize the information provided by the task execution time as a probability distribution, rather than just a simple single numerical estimate. This method aims to overcome the limitations that may occur when the traditional static scheduling strategy deals with task execution time uncertainty, and more accurately captures the random characteristics of task execution time through simulation and sampling techniques, and proposes a deadlock avoidance strategy in the case of multiple task flows with the possibility of deadlock.

[0061] The present application has been proved to be efficient and feasible through experiments, simulations and uses.

[0062] Embodiment 2

[0063] According to another embodiment of the present application, a distributed workflow load parallel scheduling device based on Monte Carlo is provided, as shown in Figure 7 , comprising:

[0064] The sampling unit 201 is configured to sample the execution time by using the Monte Carlo method.

[0065] The scheduling scheme acquisition unit 202 is configured to generate a plurality of scheduling schemes based on the static scheduling CPOP, and to use the CPOP algorithm to obtain the critical path and then use the Monte Carlo search tree to reduce the search time, and to screen out the scheduling schemes with long time.

[0066] The scheduling scheme selection unit 203 is configured to sample the execution time, calculate the minimum scheduling time obtained according to different scheduling schemes, and select the scheduling scheme with the minimum mean value of the minimum scheduling time.

[0067] The DAG preprocessing scheme based on Monte Carlo simulation technology is proposed in the distributed workflow load parallel scheduling device based on Monte Carlo in the embodiment of the application, and the scheme aims to fully utilize the probability distribution attribute of task execution time, instead of being limited to the estimation of a single evaluation. This method aims to break through the limitations of traditional static scheduling methods in dealing with task execution time uncertainty, and more accurately grasp the randomness characteristics of task execution time through simulation and sampling analysis means, so as to realize a more optimal scheduling strategy and overall performance optimization in the decision-making process.

[0068] The DAG preprocessing scheme based on Monte Carlo simulation technology is proposed in the distributed workflow load parallel scheduling device based on Monte Carlo in the embodiment of the application, and the scheme aims to fully utilize the probability distribution attribute of task execution time, instead of being limited to the estimation of a single evaluation. This method aims to break through the limitations of traditional static scheduling methods in dealing with task execution time uncertainty, and more accurately grasp the randomness characteristics of task execution time through simulation and sampling analysis means, so as to realize a more optimal scheduling strategy and overall performance optimization in the decision-making process. Secondly, in order to avoid the deadlock problem caused by resource squeeze in the multi-workflow scene, the application designs a deadlock solving algorithm.

[0069] The technical scheme of the application is described in detail as follows:

[0070] Monte Carlo Tree Search (MCTS) as an effective heuristic search technique, combines the universality of random sampling and the accuracy of tree structure search, and has made breakthrough progress in game AI field, especially in various game environments such as Go, Texas Hold'em and the like. Its operation mechanism revolves around five key steps in a cycle:

[0071] Selection: Starting from the root node, a certain strategy (such as UCB formula) is selected to select the child node for recursive search. This process will go down along the tree path until it encounters an unexpanded node or reaches the preset search depth. The goal of the selection strategy is to find the most promising child node among the explored nodes, while maintaining a certain degree of exploration. For example, the UCT formula combines the cumulative reward and visit count of the node to balance exploration and utilization.

[0072] Expansion: If the selected node is not fully expanded (i.e. there are unexplored actions), an unexpanded action is selected for expansion to generate a new child node. The goal of expansion is to expand the search space to discover new possibilities. For example, in chess games, expansion can represent trying different drop positions.

[0073] Simulation: Starting from the newly generated child node, a full random simulation (also called a "rollout") is performed. No search strategy is used during this process, but instead a simple random strategy or default strategy is used to make decisions until an end condition is reached (such as the end of the game or the maximum number of moves). The goal of the simulation is to evaluate the potential value of the current node so that better decisions can be made in subsequent searches. For example, in a chess game, a simulation could mean randomly placing pieces and then playing a few random moves to observe the final outcome.

[0074] Back propagation: Backpropagating the results of a simulation (usually the final reward) back to the parent node to update each node's statistics, such as the number of visits and cumulative reward. The goal of feedback is to gather more information so that more accurate decisions can be made in future searches. For example, in a chess game, feedback can mean feeding the simulated win or loss results back to the parent node to update its cumulative reward and number of visits.

[0075] Repeat: Return to the selection phase and continue the next round of selection, expansion, simulation, and feedback until the preset number of searches is reached or other stopping conditions are met. The goal of repetition is to improve the search quality through multiple iterations and find a solution that is closer to the optimal solution. For example, in chess games, repetition can mean multiple rounds of search to find the best placement. Simply put, it is to use the Monte Carlo method to sample the execution time and generate multiple scheduling schemes based on the static scheduling CPOP. The CPOP algorithm uses the Monte Carlo search tree to reduce the search time after obtaining the critical path, screen out the scheduling schemes that take a long time, and select the best one from this group of schemes. Here, the execution time is also sampled, and the minimum scheduling time obtained according to different scheduling schemes is calculated, and the one with the smallest average minimum scheduling time is selected.

[0076] At the same time, in an ideal situation, if there are a lot of available resources and the DAG has data dependencies, the task scheduler can maximize the degree of parallelism. However, this ideal situation is not always feasible in reality. When resources are constrained, there may be deadlock problems depending on the model, because a task may hold the allocated resources while waiting for the subtasks to complete, and the subtasks may be blocked due to insufficient resources available for execution. For example, in Figure 2 In the example (where the two-way arrows indicate dependencies between data), if enough resources R1 are allocated along the corresponding paths for tasks A, B, E, E1, D, D2, C, H, and H1 to execute, only 10 R1 units can be released after tasks E1, D2, H, and H1 complete. If no more additional R1 resources are available, then such a number of resources is insufficient for the program to proceed, resulting in a deadlock, regardless of what happens to resource R2.

[0077] For the above situation, the present invention proposes a design to solve the deadlock, the specific design is as follows Figures 3-6 As shown, whenever there is a new DAG process or the current node of the DAG process ends a task, before executing the new task node, the capacity of the currently available resources will be obtained to establish a Monte Carlo tree for n simulations. Once the number of simulations reaches the corresponding value, if all nodes are successfully traversed, it means that the current task can be completed. Therefore, the premise of all tasks currently entering the execution queue is that the task can be completed. Therefore, there will be no situation where each occupies resources and the resources are insufficient to support the next task, thereby avoiding the deadlock caused by resource preemption. In this part, the four steps of the Monte Carlo search tree are still selection, expansion, simulation, and back propagation. In the selection stage, it is necessary to start from the root node, that is, the situation where a decision is to be made, and select a node that most urgently needs to be expanded. However, the present invention will try its best to select points that have not been selected (also the points with the largest UCT value). The expansion phase is when the selection phase ends, the present invention finds a node and an action of this node that has not yet been expanded. This action will schedule the scheduling sequence previously calculated by other workflows. At this time, the present invention will obtain the currently required resource status. If the current resources of the present invention cannot support the completion of resource scheduling, the present invention will record it as 0 in the backtracking phase. If all scheduled nodes can complete the task, the present invention will reschedule other scheduling sequences previously obtained by this workflow. The existence of UCT will enable the present invention to prioritize scheduling nodes with fewer duplicate nodes, hoping to end the deadlock check as soon as possible.

[0078] The key points and intended protection points of the present invention are:

[0079] 1. To achieve the goal of minimizing the total workflow scheduling time, the present invention designs a two-stage scheduling algorithm. First, the Monte Carlo simulation method is used to predict the set of task execution times, thereby generating the optimal task execution sequence.

[0080] 2. Secondly, based on the obtained sorting results, the results that take a long time are filtered out, and the Monte Carlo search tree technology is combined to cleverly perform time sampling prediction on the scheduling sequence in the queue.

[0081] 3. In the case of multi-task flow and data exchange between tasks, a scheduling strategy is proposed to effectively prevent the occurrence of deadlock.

[0082] Compared with the prior art, the advantages of the present invention are:

[0083] This paper proposes a Monte Carlo-based DAG (Directed Acyclic Graph) scheduling technique that leverages the information provided by task execution times as probabilistic distributions, rather than simply single numerical estimates. This approach aims to overcome the limitations of traditional static scheduling strategies when dealing with uncertainty in task execution times. Through simulation and sampling techniques, it more accurately captures the stochastic nature of task execution times and proposes a deadlock avoidance strategy for multi-task flows where deadlock is a possibility.

[0084] The present invention has been proved to be efficient and feasible through experiments, simulations and applications.

[0085] Example 3

[0086] A storage medium stores a program file capable of implementing any one of the above-mentioned Monte Carlo-based distributed workflow load parallel scheduling methods.

[0087] Example 4

[0088] A processor is used to run a program, wherein when the program is run, any one of the above-mentioned Monte Carlo-based distributed workflow load parallel scheduling methods is executed.

[0089] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0090] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the system embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0092] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0093] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0094] If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes various media that can store program codes, such as a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, etc.

[0095] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A Monte Carlo-based distributed workflow load parallel scheduling method, characterized in that: The following steps are involved: S101: Using the Monte Carlo method to sample the execution time; S102: Generate multiple scheduling solutions based on the static scheduling CPOP, use the CPOP algorithm to obtain the critical path, and then use the Monte Carlo search tree to reduce the search time and filter out scheduling solutions that take a long time; S103: Sampling the execution time, calculating the minimum scheduling time according to different scheduling schemes, and selecting the scheduling scheme with the smallest mean minimum scheduling time.

2. The Monte Carlo-based distributed workflow load parallel scheduling method according to claim 1, characterized in that: After using the CPOP algorithm to obtain the critical path, the Monte Carlo search tree is used to reduce the search time, including: Monte Carlo simulation method is used to predict the task execution time set and generate the optimal task execution sequence.

3. The Monte Carlo-based distributed workflow load parallel scheduling method according to claim 2, characterized in that: In step S102, based on the optimal task execution order, scheduling plans that take a long time are screened out, and a scheduling plan with the shortest time is selected.

4. The Monte Carlo-based distributed workflow load parallel scheduling method according to claim 1, characterized in that: The method further comprises: In the case of multi-task flow and data exchange between tasks, the Monte Carlo search tree scheduling strategy is used to prevent deadlock.

5. The Monte Carlo-based distributed workflow load parallel scheduling method according to claim 4, characterized in that: Using Monte Carlo search tree scheduling strategies to prevent deadlocks includes: Whenever a new DAG process or the current node of the DAG process ends its task, before executing the new task node, the capacity of the currently available resources is obtained and a Monte Carlo tree is established to perform n simulations. Once the number of simulations reaches the corresponding value, all nodes are traversed.

6. The Monte Carlo-based distributed workflow load parallel scheduling method according to claim 5, characterized in that: The four steps of the Monte Carlo search tree include selection, expansion, simulation, and backpropagation.

7. The Monte Carlo-based distributed workflow load parallel scheduling method according to claim 6, characterized in that: In the selection phase, start from the root node of the situation where a decision is to be made and select the node that most urgently needs to be expanded, and try to choose a point that has not been selected before.

8. The Monte Carlo-based distributed workflow load parallel scheduling method according to claim 7, characterized in that: In the expansion phase, a node is found, along with an action that has not yet been expanded. Based on this action, the previously calculated scheduling sequence of other workflows is scheduled, and the currently required resource status is obtained.

9. The Monte Carlo-based distributed workflow load parallel scheduling method according to claim 8, characterized in that: When obtaining the currently required resource status, if the current resources cannot support the completion of resource scheduling, it will be recorded as 0 in the backtracking stage. If all scheduled nodes can complete the task, other scheduling sequences previously obtained by this workflow will be rescheduled.

10. A Monte Carlo-based distributed workflow load parallel scheduling device, characterized in that: include: A sampling unit, used to sample the execution time using the Monte Carlo method; The scheduling solution acquisition unit is used to generate multiple scheduling solutions based on the static scheduling CPOP. After obtaining the critical path using the CPOP algorithm, it uses the Monte Carlo search tree to reduce the search time and filter out scheduling solutions that take a long time. The scheduling scheme selection unit is used to sample the execution time, calculate the minimum scheduling time according to different scheduling schemes, and select the scheduling scheme with the smallest mean minimum scheduling time.

Images