Task scheduling method and device, computer program product and storage medium

Abstract

The invention discloses a task scheduling method and device, a computer program product and a storage medium, and relates to the technical field of cloud services, and the method comprises the steps: building a directed acyclic graph comprising a plurality of tasks, and determining the sorting value of each task; sorting the tasks based on the sorting value to obtain a task sorting result; before any task is executed according to the task sorting result, the task is used as a target task, and the scheduling value of each processing unit in the processing unit set for the target task is determined; the processing unit with the lowest scheduling value in the processing units with the available processing resources supporting execution of the target task serves as the optimal processing unit for executing the target task, and when the optimal processing unit is not the processing unit with the lowest scheduling value in all the processing units, the sorting value of the target task is determined again; and adjusting the execution sequence of the target task based on the re-determined sorting value of the target task. By applying the scheme of the invention, task scheduling can be effectively realized, and the overall operation time consumption of the task is reduced.

Description

Technical Field

[0001] This application relates to the technical field of cloud services, and particularly to a task scheduling method, device, computer program product, and storage medium. Background Art

[0002] With the continuous development of technologies such as deep learning, cloud computing has been widely applied in various industries and its popularity is still increasing. Through cloud computing, computing resources can be provided to users in the form of network services, enabling users not to provide their own computing resources but to obtain the required computing resources and services at any time and anywhere. Therefore, more and more tasks with large computing volumes and high complexities are arranged on cloud computing platforms.

[0003] The running of user tasks not only consumes a large amount of computing resources but also has extremely strict requirements for time. If user tasks can be run quickly, it can not only improve the user experience but also increase resource utilization and reduce energy consumption. Currently, different cloud computing vendors have various deployment schemes when selecting nodes for user task deployment, with different focuses, but they all need to be improved in terms of quickly completing user tasks. For example, some schemes obtain an approximate optimal solution in the direction of improving resource utilization, and some schemes, although considering the quick completion of tasks, obtain a local optimal solution for the task itself, that is, they do not consider the correlation between different tasks, resulting in the overall task execution time not being optimal.

[0004] In summary, how to effectively implement task scheduling and reduce the overall running time of tasks is a technical problem that needs to be solved urgently by those skilled in the art in the current field. Summary of the Invention

[0005] This application provides a task scheduling method, device, computer program product, and storage medium to effectively implement task scheduling and reduce the overall running time of tasks.

[0006] In a first aspect, this application provides a task scheduling method, including:

[0007] Establish a directed acyclic graph including multiple tasks, and determine the sorting values of each task in the directed acyclic graph;

[0008] Sort each task in the directed acyclic graph based on the sorting values to obtain a task sorting result, and sequentially execute each task according to the task sorting result;

[0009] Before executing any one of the tasks according to the task sorting result, use the task as the target task and determine the scheduling value of each processing unit in the set of processing units for the target task; wherein, the scheduling value represents the time taken for all tasks in each task path containing the target task to be completed when the corresponding processing unit executes the target task in the directed acyclic graph.

[0010] Use the processing unit with the lowest scheduling value among the processing units that support the execution of the target task with available processing resources as the optimal processing unit for executing the target task, and when the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units, re-determine the sorting value of the target task.

[0011] For the unexecuted part of the task sorting result, adjust the execution order of the target task based on the re-determined sorting value of the target task.

[0012] On the other hand, establish a directed acyclic graph including multiple tasks and determine the sorting value of each task in the directed acyclic graph, including:

[0013] Establish a directed acyclic graph including multiple tasks and determine the initial sorting value of each task in the directed acyclic graph through a preset sorting rule.

[0014] Adjust the initial sorting value of the corresponding task according to the weight coefficient of each task to obtain the sorting value of each task in the directed acyclic graph.

[0015] On the other hand, establish a directed acyclic graph including multiple tasks and determine the initial sorting value of each task in the directed acyclic graph through a preset sorting rule, including:

[0016] Establish a directed acyclic graph including multiple tasks and determine the initial sorting value of each task in the directed acyclic graph through a preset sorting rule, and for any one task, the initial sorting value of the task is the time taken for all the remaining tasks except the prerequisite tasks of the task to be completed in each task path containing the task in the directed acyclic graph.

[0017] On the other hand, adjust the initial sorting value of the corresponding task according to the weight coefficient of each task to obtain the sorting value of each task in the directed acyclic graph, including:

[0018] For any one task, use the out-degree of the task as the weight coefficient of the task, and use the product of the weight coefficient and the initial sorting value of the task as the obtained sorting value of the task.

[0019] On the other hand, before executing any task according to the task sorting result, taking the task as a target task, and determining the scheduling value of each processing unit in the processing unit set for the target task, including:

[0020] Before executing any task according to the task sorting result, taking the task as a target task;

[0021] For any processing unit in the processing unit set, determining the earliest completion time of the target task when the target task is executed by the processing unit, as the forward-looking evaluation time of the processing unit for the target task;

[0022] Determining the time from the completion of the execution of the target task to the completion of all subsequent tasks of the target task in each task path containing the target task in the directed acyclic graph when the target task is executed by the processing unit, as the backward-looking evaluation time of the processing unit for the target task;

[0023] Taking the sum of the forward-looking evaluation time and the backward-looking evaluation time as the determined scheduling value of the processing unit for the target task.

[0024] On the other hand, for any processing unit in the processing unit set, determining the earliest completion time of the target task when the target task is executed by the processing unit, as the forward-looking evaluation time of the processing unit for the target task, including:

[0025] For any processing unit in the processing unit set, determining the execution time of the target task when the target task is executed by the processing unit;

[0026] Determining the earliest start time of the target task when the target task is executed by the processing unit;

[0027] Summing the execution time of the target task and the earliest start time of the target task to obtain the earliest completion time of the target task, and taking it as the forward-looking evaluation time of the processing unit for the target task.

[0028] On the other hand, determining the earliest start time of the target task when the target task is executed by the processing unit, including:

[0029] Determining each parent task in the directed acyclic graph that points to the target task;

[0030] For any one of the parent tasks, determine the communication time between the parent task and the target task when the target task is executed by the processing unit, and sum the communication time and the actual completion time of the parent task to obtain the earliest start time of the target task corresponding to the parent task; wherein, the actual completion time of the parent task represents: in the directed acyclic graph, among the task paths containing the parent task, the time when the parent task and all its prerequisite tasks are completed;

[0031] Among the earliest start times of the target task corresponding to each parent task obtained, select the maximum value as the earliest start time of the target task when the target task is executed by the processing unit.

[0032] On the other hand, determine the time from when the target task is completed to when all the subsequent tasks of the target task in the task paths containing the target task in the directed acyclic graph are completed when the target task is executed by the processing unit, as the backward evaluation time of the processing unit for the target task, including:

[0033] By recursively in the way of pushing from the exit task in the directed acyclic graph to the target task, obtain the backward evaluation time of the processing unit for the target task;

[0034] wherein, represents the exit task, represents the k-th processing unit, represents the task execution time when the exit task is executed by the k-th processing unit ; represents the exit task corresponding to the k-th processing unit overhead; represents the i-th round of recursive task in the recursive process and is not the exit task, represents the task is the subtask of task ; represents the task the set of overheads composed of the overheads corresponding to each processing unit, represents the task and task communication time between; represents the task execution time when the task is executed by the k-th processing unit, max represents taking the maximum value, represents the task The overhead corresponding to the k-th processing unit indicates the time taken for the k-th processing unit to perform a backward-looking evaluation for the task ; both k and i are positive integers.

[0035] On the other hand, when the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units, re-determining the sorting value of the target task includes:

[0036] When the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units, re-determine the time taken for the backward-looking evaluation of the target task;

[0037] Add the re-determined time taken for the backward-looking evaluation of the target task to the time taken for the target task to be executed by the optimal processing unit, and adjust the obtained result according to the weight coefficient of the target task to obtain the re-determined sorting value of the target task.

[0038] On the other hand, it further includes:

[0039] Receiving a directed acyclic graph update instruction;

[0040] Based on the directed acyclic graph update instruction, perform a structural update on the established directed acyclic graph, and determine whether the update result meets the set constraints;

[0041] If not, cancel the current structural update of the directed acyclic graph and output a prompt message.

[0042] On the other hand, determining whether the update result meets the set constraints includes:

[0043] For any task in the update result, if starting from the task and returning to the task after passing through several tasks, it is determined that the update result does not meet the set constraints.

[0044] On the other hand, it further includes:

[0045] Obtain the processing resource usage of each processing unit in the processing unit set;

[0046] When the processing resource usage of any one processing unit meets the set reminder rule, output a prompt message carrying the processing resource usage of the processing unit.

[0047] On the other hand, the reminder rule includes:

[0048] The average usage of any one processing resource of the processing unit exceeds the upper threshold set for this processing resource, or is lower than the lower threshold set for this processing resource.

[0049] On the other hand, taking the processing unit with the lowest scheduling value among the processing units that support the execution of the target task with available processing resources as the optimal processing unit for executing the target task includes:

[0050] Arrange each processing unit in ascending order of the scheduling value to obtain a processing unit selection list for the target task;

[0051] Judge whether the available processing resources of the x-th processing unit in the processing unit selection list of the target task support the execution of the target task;

[0052] If not, add 1 to x and return to execute the operation of judging whether the available processing resources of the x-th processing unit in the processing unit selection list support the execution of the target task;

[0053] If so, take the processing unit corresponding to the x-th scheduling value as the optimal processing unit;

[0054] Wherein, x is a preset parameter and the initial value is 1.

[0055] On the other hand, judging whether the available processing resources of the x-th processing unit in the processing unit selection list of the target task support the execution of the target task includes:

[0056] Judge whether the central processing unit resources, memory resources, and hard disk resources of the x-th processing unit in the processing unit selection list of the target task all support the execution of the target task;

[0057] If all support the execution of the target task, determine that the processing unit supports the execution of the target task, otherwise determine that the processing unit does not support the execution of the target task.

[0058] On the other hand, after judging that the available processing resources of the x-th processing unit in the processing unit selection list do not support the execution of the target task, it further includes:

[0059] Overlay the waiting duration of the target task with the overhead corresponding to the x-th processing unit of the target task, and judge whether the overlay result is less than or equal to the overhead corresponding to the (x + 1)-th processing unit of the target task;

[0060] If so, place the target task in the waiting queue of the x-th processing unit, and take the x-th processing unit as the optimal processing unit;

[0061] If not, add 1 to x and return to execute the operation of judging whether the available processing resources of the x-th processing unit in the processing unit selection list support the execution of the target task;

[0062] Among them, the overhead corresponding to the target task for the x-th processing unit represents: when the target task is executed by the x-th processing unit, the time consumption from the start of the execution of the target task to the completion of the execution of all the subsequent tasks of the target task in each task path in the directed acyclic graph that contains the target task.

[0063] On the other hand, for the unexecuted part of the task sorting result, based on the sorting value re-determined for the target task, the adjustment of the execution order of the target task is performed, including:

[0064] For the unexecuted part of the task sorting result, a task set participating in the adjustment is determined;

[0065] Based on the sorting value re-determined for the target task, the execution order of the target task is adjusted, and the tasks participating in the adjustment are all tasks in the task set;

[0066] Among them, the task set participating in the adjustment includes the target task, and each task in the task set participating in the adjustment is continuously arranged in the task sorting result, and each task in the task set participating in the adjustment is at the same level in the directed acyclic graph.

[0067] In a second aspect, the present invention provides a task scheduling device, including:

[0068] A memory for storing a computer program;

[0069] A processor for implementing the steps of the task scheduling method as described above when executing the computer program.

[0070] In a third aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and wherein the computer program implements the steps of the task scheduling method as described above when executed by a processor.

[0071] In a fourth aspect, the present invention provides a computer program product, including a computer program, and the computer program implements the steps of the task scheduling method as described above when executed by a processor.

[0072] Applying the technical solution provided by the embodiments of the present invention, on the basis of arranging the tasks in order, the processing units are allocated with the goal of minimizing the overall task execution time. Further, the task order will be dynamically feedback-adjusted to effectively shorten the overall running time of the tasks. Specifically, there are association relationships between tasks, so a directed acyclic graph including multiple tasks needs to be established to reflect such association relationships, and the sorting values of each task in the directed acyclic graph are determined to achieve preliminary sorting, so that all DAG tasks can be executed compactly, reducing the execution time. The DAG tasks described here are also the tasks in the directed acyclic graph. After sorting each task in the directed acyclic graph based on the sorting values and obtaining the task sorting result, each task can be executed in sequence according to the task sorting result. Before executing any task according to the task sorting result, task scheduling is required, that is, determining the most suitable processing unit. And in the solution of this application, when determining the most suitable processing unit for it, not only the task itself is considered, but the processing units are allocated with the goal of minimizing the overall task execution time. Specifically, the scheduling values of each processing unit for the target task are determined. The scheduling value represents the time taken for all tasks in each task path including the target task in the directed acyclic graph to be executed when the target task is executed by the corresponding processing unit. Therefore, the processing unit with the lowest scheduling value is the most suitable processing unit for the target task, which can minimize the overall execution time of the DAG tasks.

[0073] Moreover, this application further takes into account the possible situation of resource conflicts. Therefore, after obtaining the scheduling values of each processing unit for the target task, the processing unit with the lowest scheduling value among the processing units that can support the execution of the target task with the available processing resources needs to be used as the optimal processing unit for executing the target task. It can be understood that if the optimal processing unit is the processing unit with the lowest scheduling value among all processing units, then directly use the processing unit with the lowest scheduling value among all processing units as the optimal processing unit and execute the target task. However, when the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units due to resource conflicts, the solution of this application takes into account the change of such processing units, which may affect the scheduling of subsequent tasks. Therefore, the sorting value of the target task will be re-determined, and then for the unexecuted part of the task sorting result, based on the re-determined sorting value of the target task, the execution order of the target task is adjusted to achieve the optimization of task sorting and effectively shorten the overall running time of the tasks. In summary, the solution of this application can effectively achieve task scheduling and reduce the overall running time of the tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0075] Figure 1 It is the flowchart of an embodiment of a task scheduling method provided by the embodiment of the present application;

[0076] Figure 2 It is a directed acyclic graph established in the embodiment of the present application;

[0077] Figure 3 It is another directed acyclic graph established in the embodiment of the present application;

[0078] Figure 4 It is a schematic diagram of the selection list of processing units for the target task obtained in the embodiment of the present application;

[0079] Figure 5 It is a schematic diagram of the task sorting result obtained in the embodiment of the present application;

[0080] Figure 6 It is a schematic diagram of the structure of a task scheduling device provided by the embodiment of the present application;

[0081] Figure 7 It is a schematic diagram of the structure of a computer-readable storage medium of the present invention. Detailed implementation manners

[0082] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0083] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0084] To enable those skilled in the art of the present technology to better understand the solution of the present application, the following will further describe the present application in detail in conjunction with the accompanying drawings and specific implementation manners. Please refer toFigure 1 , which is the implementation flowchart of a task scheduling method provided by an embodiment of this application. The task scheduling method may include the following steps:

[0085] Step S101: Establish a directed acyclic graph including multiple tasks, and determine the sorting values of each task in the directed acyclic graph.

[0086] A DAG (Directed Acyclic Graph) is a graph theory data structure. If a directed graph cannot return to a point after passing through several edges starting from any vertex, then this graph is a directed acyclic graph. For easy understanding, refer to Figure 2 , which is a directed acyclic graph including multiple tasks established in a specific scenario. Figure 2 In the example of , 1 to 6 respectively represent Task 1 to Task 6, and it can be seen that the DAG graph consists of multiple tasks with a sequential execution order. The subsequent tasks must perform relevant operations only after all the preceding tasks are completed. For example, Figure 2 in the example of , Task 2 can only be executed after Task 1 is completed, Task 4 can only be executed after Task 3 is completed, and Task 6 can only be executed after both Task 2 and Task 5 are completed.

[0087] When establishing a directed acyclic graph including multiple tasks, it can be established based on the instructions of the staff or automatically established through relevant establishment procedures, which does not affect the implementation of the present invention. And it can be understood that the structure of the established directed acyclic graph should conform to the actual association relationship between relevant tasks.

[0088] After establishing the directed acyclic graph, it is necessary to determine the sorting values of each task in the directed acyclic graph. The sorting values of the tasks determine the execution order of the tasks. There are various specific ways to determine the sorting values. For example, the HEFT (Heterogeneous Earliest Finish Time) scheduling algorithm can be used to determine the sorting values of each task in the directed acyclic graph. The HEFT scheduling algorithm is well applied in task compactness, can constrain multiple tasks according to the execution cost, obtain the sorting values of each task, and thus determine the execution order of each task.

[0089] Step S102: Sort each task in the directed acyclic graph based on the sorting values to obtain a task sorting result, so as to execute each task in sequence according to the task sorting result.

[0090] After obtaining the sorting values of each task in the directed acyclic graph, the tasks in the directed acyclic graph can be sorted to obtain the task sorting result. Generally speaking, the task with a larger sorting value is executed first. Then, the tasks in the directed acyclic graph can be sorted in descending order of the sorting values to obtain the task sorting result. In practical applications, the task sorting result is usually presented in the form of a list. After obtaining the task sorting result, each task can be executed in sequence according to the task sorting result.

[0091] Step S103: Before executing any task according to the task sorting result, set the task as the target task and determine the scheduling value of each processing unit in the processing unit set for the target task; wherein, the scheduling value represents the time consumption when the corresponding processing unit executes the target task and all tasks in each task path containing the target task in the directed acyclic graph are executed.

[0092] Before executing any task according to the task sorting result, it is necessary to select a suitable processing unit to perform the scheduling of the task, that is, select a suitable processing unit to execute the task. The processing unit described in this application is a unit that can process specific tasks based on its processing resources and can be one or more physical devices of a specific type.

[0093] In the solution of this application, when determining the most suitable processing unit for a task, it is not only considered the speed of executing the task itself, but the processing unit is allocated with the goal of the shortest overall task execution time, so that the overall execution time of the DAG task is the shortest.

[0094] Set the task that currently needs to allocate a processor as the current target task, and determine the scheduling value of each processing unit in the processing unit set for the target task. In the solution of this application, the scheduling value of a certain processing unit for the target task reflects the time consumption when all tasks in each task path containing the target task are executed. Therefore, it can be seen that the lower the scheduling value, the more conducive to ensuring that the DAG task can be executed quickly as a whole. Of course, there can be various specific implementation methods for calculating the scheduling value, as long as the calculation of the scheduling value can be effectively implemented.

[0095] Step S104: Select the processing unit with the lowest scheduling value among the processing units that can support the execution of the target task with available processing resources as the optimal processing unit for executing the target task. And when the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units, re-determine the sorting value of the target task.

[0096] If there is no resource conflict, it is understandable that for the target task, the target task can be directly executed using the processing unit with the lowest scheduling value among all processing units. However, in practical applications, due to various reasons such as the emergence of time fragments during task execution, abnormal task execution that occupies the resources of a certain processing unit for a long time, resource conflicts may occur. That is, for the processing unit with the lowest scheduling value among all processing units, its current available processing resources cannot support the execution of the target task. Therefore, in the solution of this application, the processing unit with the lowest scheduling value among the processing units whose current available processing resources can support the execution of the target task can be used as the optimal processing unit for the target task.

[0097] Moreover, the solution of this application also takes into account that if the optimal processing unit of the finally determined target task is not the processing unit with the lowest scheduling value among all processing units, it means that there has been a situation of processing unit change due to available processing resource conflicts. That is, the processing unit for executing some tasks is not the most ideal processing unit originally planned. Such a situation may also affect the scheduling of subsequent tasks. Therefore, in order to effectively shorten the overall running time of tasks, for such a situation, the solution of this application will re-determine the sorting value of the target task, so that for the unexecuted part of the task sorting result, the execution order of the target task will be adjusted based on the re-determined sorting value of the target task, realizing the optimization of task sorting.

[0098] Step S105: For the unexecuted part of the task sorting result, adjust the execution order of the target task based on the re-determined sorting value of the target task.

[0099] It is understandable that the tasks in the task sorting result are executed sequentially. When a certain task is used as the target task and the aforementioned situation of processing unit change occurs, no matter how the execution order of the target task is adjusted, the execution order of the target task cannot be adjusted to before the tasks that have been executed in the task sorting result. Therefore, the adjustment in this step is for the unexecuted part of the task sorting result. Specifically, it is to adjust the execution order of the target task based on the re-determined sorting value of the target task. That is, the target task uses the re-determined sorting value, and other tasks use the sorting value obtained in the aforementioned step S101. By adjusting the execution order of the target task, the optimization of task sorting is realized to effectively shorten the overall running time of tasks.

[0100] In a specific implementation manner of the present invention, step S101 may include:

[0101] Establish a directed acyclic graph including multiple tasks, and determine the initial sorting values of the tasks in the directed acyclic graph through a preset sorting rule;

[0102] According to the respective weight coefficients of each task, adjust the initial sorting values of the corresponding tasks to obtain the sorting values of each task in the directed acyclic graph.

[0103] In this implementation manner, the initial sorting values of each task in the directed acyclic graph are determined through a preset sorting rule. The specific setting of the preset sorting rule can be set according to needs. For example, the HEFT scheduling algorithm can be used as the preset sorting rule, or the PEFT (Parameter-Efficient Fine-Tuning) scheduling algorithm can be used as the preset sorting rule, and it can be selected according to actual needs.

[0104] After determining the initial sorting values of each task in the directed acyclic graph through the preset sorting rule, in this implementation manner, the initial sorting values of the corresponding tasks will be further adjusted according to the respective weight coefficients of each task. The result after adjustment will be used as the sorting value of each task in the directed acyclic graph obtained in step S101. This implementation manner takes into account that the execution of different tasks has different degrees of influence on the execution of subsequent tasks. Therefore, in order to make the obtained sorting more reasonable and more conducive to ensuring the overall execution efficiency of DAG tasks, it is supported to adjust the initial sorting values of the corresponding tasks according to the respective weight coefficients of each task. It can be seen that such a method also improves flexibility, that is, through the setting of weight coefficients, if necessary in practical applications, the sorting of tasks can be adjusted through weight coefficients.

[0105] When adjusting the initial sorting values of the corresponding tasks according to the respective weight coefficients of each task, the specific adjustment method can be set according to needs. For example, in a specific implementation manner of the present invention, adjusting the initial sorting values of the corresponding tasks according to the respective weight coefficients of each task to obtain the sorting values of each task in the directed acyclic graph may specifically include:

[0106] For any task, take the out-degree of the task as the weight coefficient of the task, and take the product of the weight coefficient and the initial sorting value of the task as the obtained sorting value of the task.

[0107] This implementation manner takes into account that the larger the out-degree of a task, the greater the degree of influence of the task on the execution of subsequent tasks. Therefore, the out-degree of the task can be used as the weight coefficient of the task, that is, the out-degree of the task is used to measure the degree of influence of the task on the execution of subsequent tasks, and then the initial sorting value of the task is adjusted. Currently, scheduling algorithms such as HEFT usually do not consider the degree of influence of tasks on the execution of subsequent tasks.

[0108] Specifically, the out-degree of a task is also the number of subtasks of the task. For easy understanding, refer to Figure 3 , for exampleFigure 3 It is a DAG schematic diagram in a specific scenario. Figure 3 The numbers 1 to 6 in it represent Task 1 to Task 6. In this example, Task 2 has 3 subtasks, namely Task 4, Task 5, and 6.

[0109] In this implementation manner, the obtained sorting value of a task can be expressed as f_Rank(t i ) = Rank(t i ) × out(t i ). Here, Rank(t i ) represents the initial sorting value of task t i , and out(t i ) represents the out-degree of task t i , that is, its weight coefficient. For example, Figure 3 the out-degree of Task 2 in it is 3. And f_Rank(t i ) represents the sorting value of task t i in the obtained directed acyclic graph. Task t i can be any task in the directed acyclic graph.

[0110] In a specific implementation manner of the present invention, establishing a directed acyclic graph including multiple tasks and determining the initial sorting values of the tasks in the directed acyclic graph through a preset sorting rule may specifically include:

[0111] Establish a directed acyclic graph including multiple tasks, and determine the initial sorting values of the tasks in the directed acyclic graph through a preset sorting rule. And for any task, the initial sorting value of this task is the time-consuming when all the remaining tasks except the pre-task of this task are completed in each task path including the task in the directed acyclic graph.

[0112] This implementation manner takes into account that when obtaining the sorting values of the tasks in the directed acyclic graph in step S101, the sorting value calculation principle used should be the same as the sorting value calculation principle used when recalculating the sorting value of the target task in step S104. And the goal of the solution of this application is to effectively shorten the overall running time-consuming of the tasks. Therefore, when calculating the sorting value of a certain task, it should also be calculated with this goal in mind, so that the tasks in the directed acyclic graph can be reasonably sorted and the overall running time-consuming can be shortened.

[0113] In this implementation, for any task, its initial sorting value refers to the time taken for all tasks other than the prerequisite tasks of this task to be completed in all task paths in the directed acyclic graph that contain this task. That is, in this implementation, the initial sorting value of a task examines subsequent tasks, such that the longer the time taken for all subsequent tasks of this task to be completed in all task paths that contain this task, the higher the priority of this task. Taking Figure 3 task 9 in Figure 3 as an example, the task paths that contain task 9 are: task 1 to task 2 to task 6 to task 9 to task 10 to task 11, and task 3 to task 7 to task 8 to task 9 to task 10 to task 11, a total of 2. Among them, tasks 1, 2, 6, 3, 7, and 8 constitute all the prerequisite tasks of task 9, and tasks 10 and 11 constitute all the subsequent tasks of task 9. In this example, the time taken for tasks 9, 10, and 11 to be completed is the initial sorting value of task 9.

[0114] In addition, it should be noted that when performing step S101, if the scheme of first calculating the initial sorting value and then adjusting the initial sorting value based on the weight coefficient is adopted, then when performing step S104, the same principle needs to be used for calculating the sorting value. For example, when performing step S101, the time taken described in this implementation is used as the initial sorting value of the task, and it is multiplied by the out-degree of the task to obtain the sorting value of the task. Then, when performing step S104 to re-determine the sorting value of the target task, the same principle needs to be used to first calculate the time taken for all subsequent tasks of the target task to be completed in all task paths that contain the target task, and multiply it by the out-degree of the task to obtain the re-determined sorting value of the target task. Similarly, for example, when performing step S101, the out-degree of the task is not introduced, but the time taken described in this implementation is directly used as the sorting value of the task. Then, when performing step S104, the same principle needs to be used to calculate the time taken for all subsequent tasks of the target task to be completed in all task paths that contain the target task, and then directly use it as the sorting value of the target task. In short, when calculating the sorting value of a task in the solution of this application, the calculation logic is pre-determined and the same calculation logic is used for each task. If it is necessary to multiply by the out-degree during the calculation process, then the sorting value calculation of each task should be operated in this way.

[0115] In a specific implementation of the present invention, step S103 may specifically include:

[0116] Before executing any task according to the task sorting result, take this task as the target task;

[0117] For any one of the processing units in the processing unit set, determine the earliest completion time of the target task when the target task is executed by this processing unit, and use it as the forward-looking evaluation time of the processing unit for the target task;

[0118] Determine the time taken from the completion of the execution of the target task to the completion of all the subsequent tasks of the target task in each task path containing the target task in the directed acyclic graph when the target task is executed by this processing unit, and use it as the backward-looking evaluation time of the processing unit for the target task;

[0119] Take the sum of the forward-looking evaluation time and the backward-looking evaluation time as the scheduling value of the determined processing unit for the target task.

[0120] This implementation mode takes into account that the scheduling value of the target task can be measured based on the forward-looking evaluation time and the backward-looking evaluation time of the target task, which can effectively reflect the overall execution situation of the DAG task.

[0121] Specifically, the forward-looking evaluation time aims to determine the earliest completion time of the target task. This process progresses along with the execution order of the tasks, thus easily tracking the exact completion time of the predecessor tasks. The backward-looking evaluation time aims to ensure that all the subsequent tasks of the target task can be executed promptly, and its evaluation basis can be the data of the farthest distance from the final exit task. Through the forward-looking evaluation time and the forward-looking evaluation time, it can effectively reflect the time taken for the overall execution of the DAG task when a certain processing unit executes the target task.

[0122] The specific method for calculating the earliest completion time of the target task can be set according to actual needs. Usually, considering the earliest start time of the target task and the computing consumption of the target task on different processing units, the earliest completion time of the target task can be conveniently deduced.

[0123] In this regard, in a specific implementation mode of the present invention, for any one of the processing units in the processing unit set, determining the earliest completion time of the target task when the target task is executed by this processing unit and using it as the forward-looking evaluation time of the processing unit for the target task may specifically include:

[0124] For any one of the processing units in the processing unit set, determine the execution time of the target task when the target task is executed by this processing unit;

[0125] Determine the earliest start time of the target task when the target task is executed by this processing unit;

[0126] Sum the execution time of the target task and the earliest start time of the target task to obtain the earliest completion time of the target task, and use it as the forward-looking evaluation time of this processing unit for the target task.

[0127] In this implementation mode, the earliest completion time of the target task can be conveniently determined, which can be expressed as . In this formula, represents the k-th processing unit, represents task , which can be the i-th task among all tasks, and here it also refers to the target task. refers to the task execution time when the k-th processing unit executes task , is the earliest start time of task , is the forward-looking evaluation time of the k-th processing unit for task , that is, when the k-th processing unit executes the target task, the earliest completion time of task .

[0128] When the target task is executed by different processing units, the execution time of the target task will be different. The earliest start time of the target task refers to the time required for all the prerequisite tasks of the target task to be completed so that the target task can start to be executed. In this implementation mode, by summing the execution time of the target task and the earliest start time of the target task, the earliest completion time of the target task can be conveniently obtained, that is, the forward-looking evaluation time of the corresponding processing unit for the target task is obtained. In addition, it should be noted that the task execution time of a task in different processing units and the communication time between different processing units when the task is transferred between processing units can be determined in advance through methods such as experimental statistics and theoretical analysis.

[0129] There are various specific implementation methods for determining the earliest start time of the target task. For example, in a specific implementation mode of the present invention, to determine the earliest start time of the target task when the target task is executed by this processing unit, it can specifically include:

[0130] Determine each parent task in the directed acyclic graph that points to the target task;

[0131] For any parent task, determine the communication time between the parent task and the target task when the target task is executed by this processing unit, and sum the communication time and the actual completion time of the parent task to obtain the earliest start time of the target task corresponding to the parent task; among them, the actual completion time of the parent task represents: in the directed acyclic graph, in each task path containing the parent task, the time when the parent task and all its prerequisite tasks are completed;

[0132] Among the earliest start times of the target task corresponding to each parent task obtained, select the maximum value as the earliest start time of the target task determined when the target task is executed by this processing unit.

[0133] Still taking the task that is the current target task as an example for illustration. In this implementation, the earliest start time is obtained in the following way. In this formula, refers to the actual completion time of task , that is, from the start of the execution of the first task in the directed acyclic graph to the completion of the execution of task , the elapsed time, and is the communication time between task and task . represents that task is the parent node of task . is the earliest start time of task when executed by the k-th processing unit for task .

[0134] Refer to Figure 2 . When task 1 is the current target task, for task 1 in Figure 2 , since it has no parent task, the earliest start time ESP (Earliest Start time Point) of task 1 can be directly determined to be 0.

[0135] When task 2 is the current target task, for task 2 in Figure 2 , it has only 1 parent task, that is, task 1. For example, the total number of processing units is 10, and taking the look-ahead evaluation time of the 1st processing unit for the target task as an example. When determining the earliest start time ESP of task 2, the communication time between task 1 as the parent task and task 2 when the 1st processing unit executes task 2 can be determined. Adding this communication time to 0 gives the earliest start time of task 2 corresponding to task 1 when the 1st processing unit executes task 2. Since task 2 has only 1 parent task, this value is also the earliest start time of task 2 determined when the 1st processing unit executes task 2. Since there are 10 processing units in this example, it can be understood that in the example of Figure 2 , for task 2, for each of the 10 processing units, the earliest start time of task 2 can be obtained. Combining the task execution time when the corresponding processing unit executes task 2, the look-ahead evaluation times of the 10 processing units for task 2 can be obtained respectively. That is, in this example, when task 2 is the target task, a total of 10 look-ahead evaluation times of task 2 can be obtained, corresponding to different processing units.

[0136] Taking Figure 2 Task 6 in [Example] as an example, it has 2 parent tasks, namely Task 2 and Task 5. Still taking the total number of processing units as 10, and taking the look-ahead evaluation time-consuming of the 1st processing unit for the target task as an example. When determining the earliest start time ESP of Task 6, when it is determined that Task 6 is executed by the 1st processing unit, the communication time-consuming between Task 2, which is a parent task, and Task 6 can be determined. Adding this communication time-consuming to the AFP of Task 2, the earliest start time of Task 6 corresponding to Task 2 when Task 6 is executed by the 1st processing unit is obtained. The AFP of Task 2 described here, that is, the actual completion time-consuming of Task 2, includes the task execution time-consuming of Task 1, the task execution time-consuming of Task 2, and the communication time-consuming between Task 1 and Task 2.

[0137] Similarly, when it is determined that Task 6 is executed by the 1st processing unit, the communication time-consuming between Task 5, which is a parent task, and Task 6 needs to be determined. Adding this communication time-consuming to the AFP of Task 5, the earliest start time of Task 6 corresponding to Task 5 when Task 6 is executed by the 1st processing unit is obtained. The AFP of Task 5 described here, that is, the actual completion time-consuming of Task 5, includes the task execution time-consuming of Task 3, the task execution time-consuming of Task 4, the task execution time-consuming of Task 5, the communication time-consuming between Task 3 and Task 4, and the communication time-consuming between Task 4 and Task 5.

[0138] In the above example, the earliest start time of Task 6 corresponding to Task 2 and the earliest start time of Task 6 corresponding to Task 5 are obtained. Then the maximum value can be selected as the earliest start time of Task 6 when it is determined that Task 6 is executed by Processing Unit 1. Similarly to the above text, since there are 10 processing units in this example, it can be understood that in Figure 2 the example of [Example], for Task 6, for each of the 10 processing units, the earliest start time of Task 6 can be obtained. Combining with the task execution time-consuming when the corresponding processing unit executes Task 6, the look-ahead evaluation time-consuming of each of the 10 processing units for Task 6 can be obtained. That is, in this example, when Task 6 is the target task, a total of 10 look-ahead evaluation time-consuming of Task 6 can be obtained, corresponding to different processing units.

[0139] In this implementation manner, among the earliest start times of the target task corresponding to each parent task, the maximum value is selected as the earliest start time of the target task determined when the target task is executed by the processing unit. It can be seen that in the scenario of multi-task processing, it is often the task with the longest time consumption that determines the final end time of the entire batch of tasks. Therefore, for the target task, its earliest start time needs to consider all its preceding nodes. And in practical applications, when the task execution times of individual tasks are generally the same, it is usually the longest task path that determines the earliest start time of the target task. Moreover, this implementation manner calculates the earliest start time of the target task, taking into account the communication time consumption between the parent task and the child task. In summary, this implementation manner can conveniently determine the earliest start time of the target task, and the obtained earliest start time of the target task is also relatively accurate.

[0140] In a specific implementation manner of the present invention, when it is determined that the target task is executed by the processing unit, the time consumption from the completion of the execution of the target task to the completion of the execution of all subsequent tasks of the target task in each task path including the target task in the directed acyclic graph is determined as the backward evaluation time consumption of the processing unit for the target task, including:

[0141] By the method of recursively deriving from the exit task in the directed acyclic graph to the target task, the backward evaluation time consumption of the processing unit for the target task is obtained;

[0142] Wherein, represents the exit task, represents the k-th processing unit, represents the task execution time consumption when the exit task is executed by the k-th processing unit, represents the exit task corresponding to the k-th processing unit of the overhead; represents the task , which can be the i-th round of recursive task in the recursive process and is not the exit task, represents the task is the sub-task of the task , represents the task corresponding to the overhead set composed of the overheads of each processing unit, represents the task and the task between the communication time consumption, represents the task execution time consumption when the task is executed by the k-th processing unit, It represents the task corresponding to the overhead of the k-th processing unit, where max means taking the maximum value It represents the backward-looking evaluation time of the k-th processing unit for the task Both k and i are positive integers

[0143] Specifically, the backward-looking evaluation time of this implementation is represented by LDD (Longest Distance Data). For the sake of easy understanding, still taking Figure 2 as an example. For instance, if task 6 is the current target task, then for task 6, it belongs to the exit task, that is, task 6 has no subtasks. Therefore, the set of overheads corresponding to each processing unit for task 6 can be directly calculated through Taking the example that there are 10 processing units in total, the set of overheads (rank set) of task 6 includes 10 values corresponding to 10 different processing units. And LDD reflects the longest overhead from the target task to the exit task, and does not include the target task execution time of the target task itself. Therefore, for task 6, the backward-looking evaluation time LDD of each processing unit for task 6 is 0

[0144] Taking another example Figure 2 where task 2 in is the current target task. Then for task 2, there is only 1 task path including task 2, and task 2 does not belong to the exit task. Then, by recursively calculating from bottom to top through task 6 as the exit task, through the set of overheads of task 2 can be obtained. Specifically, taking the 1st processing unit among 10 processing units as an example, and for the sake of easy description, when performing the calculation of

[0145] Similarly, if Figure 2If Task 1 in it is the current target task, then for Task 1, if Task 1 is executed by Processing Unit No. 1, based on the overhead set of Task 2 determined above (including 10 values corresponding to 10 different processing units), combined with the task execution time of Task 1 when different processing units execute Task 1, and the communication time between Task 1 and Task 2 (the communication time between each processing unit executing Task 2 and Processing Unit No. 1 executing Task 1), 10 different rank values in the case of Task 1 being executed by Processing Unit No. 1 can be obtained. Taking the maximum value among them is the finally determined rank value in the case of Task 2 being executed by Processing Unit No. 1. In addition, it can be understood that Figure 2 In the example of Figure 2 , if Task 1 has only 1 subtask, and if Task 1 has other subtasks, the rank values of the corresponding subtasks can also be obtained through recursion, and by taking the maximum value, the rank value in the case of Task 1 being executed by Processing Unit No. 1 can finally be obtained. Since there are 10 processing units in total as an example, based on the same principle, calculations for other processing units are carried out, and finally the overhead set (rank set) of Task 1 can be obtained. This overhead set also includes 10 rank values, corresponding to 10 different processing units respectively, that is, the overheads (rank values) of Task 1 corresponding to each of the 10 processing units respectively.

[0146] After obtaining the overhead set (rank set) of the target task, for any one processing unit, subtracting the target task execution time of the target task when executed by this processing unit from the rank value corresponding to this processing unit can obtain the backward-looking evaluation time of this processing unit for the target task, that is, the backward-looking evaluation time LDD when this processing unit executes the target task.

[0147] In a specific embodiment of the present invention, when the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units, re-determine the sorting value of the target task, including:

[0148] When the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units, re-determine the backward-looking evaluation time of the target task;

[0149] Add the re-determined backward-looking evaluation time of the target task to the target task execution time of the target task when executed by the optimal processing unit, and adjust the obtained result according to the weight coefficient of the target task to obtain the re-determined sorting value of the target task.

[0150] For the case where the sorting value of the target task needs to be re-determined, this implementation mode takes into account that the sorting value can be calculated based on the rank value. Specifically, based on the above description, for a certain processing unit, adding the backward evaluation time LDD when the processing unit executes the target task to the target task execution time when the processing unit executes the target task can obtain the overhead (rank value) in the case where the processing unit executes the target task. Determining the execution order of tasks based on the magnitude of this rank value can effectively ensure the compactness of task execution and reduce the overall execution time of DAG tasks. Therefore, in this implementation mode, the calculation of the sorting value is achieved according to the rank value when the processing unit executes the target task.

[0151] It should be noted that in some cases, the rank value when the processing unit executes the target task can be directly used as the sorting value of the target task. However, this implementation mode takes into account that in order to ensure flexibility and the rationality of the obtained sorting value, it can be further adjusted in combination with a weight coefficient, and the result after adjustment is used as the sorting value of the target task. For example, referring to the above description, the weight coefficient here can specifically use the out-degree of the target task. And as described above, if the out-degree of the target task is used for the calculation of the sorting value, then for each task, whether the sorting value is calculated in step S101 or step S104, the out-degree of the target task will be used for the calculation of the sorting value.

[0152] In a specific implementation mode of the present invention, it may further include:

[0153] Receiving a directed acyclic graph update instruction;

[0154] Based on the directed acyclic graph update instruction, performing a structural update on the established directed acyclic graph and determining whether the update result meets the set constraints;

[0155] If not, canceling the structural update of the directed acyclic graph this time and outputting a prompt message.

[0156] This implementation mode takes into account that due to reasons such as business adjustment, it may be necessary to update the directed acyclic graph. Therefore, the structure of the established directed acyclic graph can be updated based on the directed acyclic graph update instruction. And this implementation mode further takes into account that the directed acyclic graph update instruction is usually output by the staff and may be incorrect, that is, based on the directed acyclic graph update instruction, performing a structural update on the established directed acyclic graph may result in a directed acyclic graph with errors. Therefore, in this implementation mode, it will be determined whether the update result meets the set constraints. If not, it can be determined that the update of the directed acyclic graph this time is incorrect, for example, caused by the misoperation of the staff. Therefore, the structural update of the directed acyclic graph this time will be canceled, and in order to remind the staff, a prompt message will be output.

[0157] In a specific embodiment of the present invention, determining whether the update result meets the set constraints may include: for any task in the update result, when starting from the task and returning to the task after several tasks, it is determined that the update result does not meet the set constraints.

[0158] When determining whether the update result meets the set constraints, the specific constraint rules used can be set and adjusted according to actual needs, and may include one or more constraint rules. This embodiment takes into account that for any task in the update result, when starting from the task and returning to the task after several tasks, it indicates that it is no longer a directed acyclic graph but a cyclic graph. Therefore, it can be determined that the update result does not meet the set constraints. In other specific scenarios, other constraints can also be set according to actual needs. For example, it is restricted that a certain task must follow another specified task immediately, or it is restricted that a certain task must be the starting task, etc.

[0159] In a specific embodiment of the present invention, it may further include:

[0160] Obtain the processing resource usage of each processing unit in the processing unit set;

[0161] When the processing resource usage of any one processing unit meets the set reminder rule, output a reminder message carrying the processing resource usage of the processing unit.

[0162] The solution of this application can effectively implement task scheduling. Moreover, in an ideal situation, the fewer resource conflict situations occur, that is, in an ideal situation, the resources of each processing unit should be reasonably applied. In this embodiment, the processing resource usage of each processing unit in the processing unit set will be obtained. If the processing resource usage of any one processing unit meets the set reminder rule, it means that the processing resource usage of this processing unit may be abnormal. Therefore, a reminder message carrying the processing resource usage of this processing unit can be output to remind the staff to pay attention to this processing unit. For example, the staff can then check whether the processing unit is not assigned tasks because the processing resources of this processing unit are set unreasonably, or is often assigned too many tasks. Of course, it can also be checked whether there are other reasons for the abnormal processing resource usage, such as checking whether the communication time consumption between this processing unit and other processing units is normal, etc.

[0163] In a specific embodiment of the present invention, the reminder rule includes:

[0164] The average usage of any one processing resource of the processing unit exceeds the upper threshold set for this processing resource, or is lower than the lower threshold set for this processing resource.

[0165] If the usage of the processing resources of any one processing unit conforms to the set reminder rule, it indicates that the usage of the processing resources of this processing unit may be abnormal. Therefore, when setting the reminder rule, it should be able to effectively reflect that the usage of the processing resources of the processing unit is abnormal. This implementation mode takes into account that the processing resources of the processing unit may include one or more processing resources, such as CPU resources, memory resources, network resources, hard disk resources, etc. The processing resources of different processing units may also be different. In this regard, in this implementation mode, when the average usage of any one processing resource exceeds the upper threshold set for this processing resource or is lower than the lower threshold set for this processing resource, it indicates that the average usage of this processing resource is too high or too low, and then it can be regarded that the usage of the processing resources of this processing unit is abnormal, that is, it is determined that the usage of the processing resources of this processing unit conforms to the set reminder rule. Of course, in other specific implementation modes, other reminder rules can also be added according to actual needs, as long as it can effectively reflect that the usage of the processing resources of the processing unit is abnormal.

[0166] In a specific implementation mode of the present invention, the processing unit with the lowest scheduling value among the processing units that use available processing resources to support the execution of the target task described in step S104 is used as the optimal processing unit for executing the target task, which may specifically include:

[0167] Arrange each processing unit in ascending order of the scheduling value to obtain a processing unit selection list for the target task;

[0168] Judge whether the available processing resources of the x-th processing unit in the processing unit selection list of the target task support the execution of the target task;

[0169] If not, add 1 to x and return to execute the operation of judging whether the available processing resources of the x-th processing unit in the processing unit selection list support the execution of the target task;

[0170] If so, use the processing unit corresponding to the x-th scheduling value as the optimal processing unit;

[0171] Wherein, x is a preset parameter and the initial value is 1.

[0172] For easy understanding, refer to Figure 4 Taking task 1 as the current target task as an example, after determining the scheduling values of each processing unit in the processing unit set for task 1 respectively, for example, the value of the 1st processing unit P1 is , that is, the scheduling value of the 1st processing unit P1 for Task 1, is the minimum among the scheduling values of all processing units for Task 1. Therefore, the 1st processing unit P1 is at the top of the processing unit selection list for Task 1. And so on, the scheduling value of the 2nd processing unit P2 for Task 1 is the second smallest value. Therefore, the 2nd processing unit P2 is at the second position in the processing unit selection list for Task 1.

[0173] For the remaining tasks other than Task 1, before executing the task, it can be regarded as the current target task, and based on the order of the scheduling values from low to high, the processing unit selection list can be obtained. Of course, the processing unit selection lists corresponding to different tasks can be different. For example Figure 4 In the example of Task 9, the 2nd processing unit P2 is at the top of its processing unit selection list, and the 5th processing unit P5 is at the second position in its processing unit selection list.

[0174] In this example, for the processing unit selection list of Task 1, first, it is necessary to determine whether the available processing resources of the 1st processing unit in the processing unit selection list (that is, the 1st processing unit P1 in this example) support the execution of the target task (that is, Task 1 in this example). If it supports, it means that the current available processing resources of the 1st processing unit P1 are sufficient. Therefore, the 1st processing unit in the processing unit selection list of Task 1 (that is, the 1st processing unit P1 in this example) can be directly used as the optimal processing unit for Task 1. At this time, there is no need to change the optimal processing unit.

[0175] If the available processing resources of the 1st processing unit P1 do not support the execution of the target task, then x needs to be incremented by 1, and then return to the operation of determining whether the available processing resources of the xth processing unit in the processing unit selection list support the execution of the target task. In this example, after incrementing x by 1, it is equal to 2. Then, it is necessary to determine whether the available processing resources of the processing unit corresponding to the 2nd scheduling value in the processing unit selection list of Task 1 (that is, the 2nd processing unit P2 in this example) support the execution of Task 1. And so on, finally, among the processing units whose available processing resources support the execution of Task 1, the processing unit with the lowest scheduling value can be determined. In this implementation, by sequentially judging the processing units in the processing unit selection list of the target task, the processing unit with the lowest scheduling value among the processing units whose available processing resources support the execution of the target task can be simply and conveniently determined, that is, the optimal processing unit of the target task can be simply and conveniently determined.

[0176] In a specific implementation of the present invention, determining whether the available processing resources of the xth processing unit in the processing unit selection list of the target task support the execution of the target task may specifically include:

[0177] Determine whether the CPU resources, memory resources, and hard disk resources of the x-th processing unit in the processing unit selection list for the target task all support the execution of the target task;

[0178] If all support the execution of the target task, determine that the processing unit supports the execution of the target task; otherwise, determine that the processing unit does not support the execution of the target task.

[0179] This implementation mode takes into account that the available processing resources of the processing unit can be mainly measured by the CPU resources, memory resources, and hard disk resources. Therefore, when determining whether the available processing resources of the x-th processing unit in the processing unit selection list support the execution of the target task, specifically, the judgment of these three resources is carried out, and it can be more convenient and accurate to determine whether the processing unit supports the execution of the target task. Of course, in other specific implementation modes, the determination criteria can be adjusted according to needs.

[0180] In a specific implementation mode of the present invention, after determining that the available processing resources of the x-th processing unit in the processing unit selection list do not support the execution of the target task, it may further include:

[0181] Add the waiting duration of the target task to the overhead corresponding to the x-th processing unit of the target task, and determine whether the added result is less than or equal to the overhead corresponding to the (x + 1)-th processing unit of the target task;

[0182] If so, place the target task in the waiting queue of the x-th processing unit, and take the x-th processing unit as the optimal processing unit;

[0183] If not, increment x by 1 and return to execute the operation of determining whether the available processing resources of the x-th processing unit in the processing unit selection list support the execution of the target task;

[0184] Among them, the overhead corresponding to the x-th processing unit of the target task refers to: when the target task is executed by the x-th processing unit, from the start of the execution of the target task to the completion of the execution of all subsequent tasks of the target task in each task path containing the target task in the directed acyclic graph.

[0185] This implementation further takes into account that after it is determined that the available processing resources of the x-th processing unit in the processing unit selection list do not support the execution of the target task, in addition to directly judging the next processing unit in the processing unit selection list, the target task can also be selected to wait. This is because the available processing resources of the processing unit are dynamically changing. If the waiting duration of the target task is superimposed on the overhead of the target task corresponding to the x-th processing unit, and the resulting value is less than or equal to the overhead of the target task corresponding to the (x + 1)-th processing unit, it means that compared with switching to the (x + 1)-th processing unit to execute the target task, the target task should still wait. After the processing resources of the x-th processing unit are released, the x-th processing unit is used as the optimal processing unit to execute the target task, which is beneficial to reducing the overall time consumption of the DAG task. That is to say, even if the waiting duration of the target task is included, the x-th processing unit is still the most suitable processing unit for the target task. Therefore, in this implementation, in such a situation, the x-th processing unit will be used as the optimal processing unit.

[0186] It can be seen that this implementation needs to evaluate whether to switch the processing unit for executing the target task based on the waiting duration of the target task, that is, it is necessary to judge whether it holds. This formula takes the task as an example of the target task. is the waiting duration of the target task. This waiting duration can be obtained by summing the remaining time of the task currently being executed by the x-th processing unit and the task execution durations of the tasks not executed in the waiting queue of the x-th processing unit. That is, the overhead of the target task corresponding to the x-th processing unit in the processing unit selection list is equivalent to the rank value described above, which represents the time consumption from the start of the execution of the target task to the completion of all the subsequent tasks of the target task in each task path containing the target task in the directed acyclic graph when the target task is executed by the x-th processing unit in the processing unit selection list. Simply put, the rank value of a task corresponding to a certain processing unit refers to the task execution duration of the task executed by this processing unit and the maximum time consumption for all the subsequent tasks of this task to be completed. As described in an implementation above, it can be specifically calculated recursively from the exit task from bottom to top. That is, the overhead of the target task corresponding to the (x + 1)-th processing unit in the processing unit selection list represents the time consumption from the start of the execution of the target task to the completion of all the subsequent tasks of the target task in each task path containing the target task in the directed acyclic graph when the target task is executed by the (x + 1)-th processing unit in the processing unit selection list.

[0187] In a specific implementation manner of the present invention, it may further include: statistically counting the change situation of the processing unit to obtain the occurrence times of the change situation of the processing unit within a unit time period, and the occurrence probability of the change situation of each processing unit within a unit time period.

[0188] The change situation of the processing unit described in this implementation manner, that is, the optimal processing unit of the determined target task, is not the processing unit with the lowest scheduling value. If there is a change situation of the processing unit, it indicates that there is a resource conflict. Ideally, the resource conflict situation should not be excessive. Therefore, in this implementation manner, the change situation of the processing unit will be statistically counted to obtain the occurrence times of the change situation of the processing unit within a unit time period. For example, the occurrence times of the change situation of the processing unit within 1 day are obtained. If the occurrence times of the change situation of the processing unit within a unit time period are excessive, the staff can be reminded for the staff to analyze and process. In addition, this implementation manner will also statistically count the occurrence probability of the change situation of each processing unit within a unit time period. If the occurrence probability of a certain processing unit having a change situation of the processing unit is very high, the staff can also be reminded to analyze and process. For example, the staff can subsequently check whether the processing resources of this processing unit are set unreasonably, and can check whether the communication time consumption between this processing unit and other processing units is normal, etc.

[0189] In a specific implementation manner of the present invention, for the unexecuted task part in the task sorting result, based on the sorting value re-determined for the target task, the execution order of the target task is adjusted, including:

[0190] For the unexecuted task part in the task sorting result, determine the task set participating in the adjustment;

[0191] Based on the sorting value re-determined for the target task, adjust the execution order of the target task, and all the tasks participating in the adjustment are tasks in the task set.

[0192] Among them, the task set participating in the adjustment includes the target task, and each task in the task set participating in the adjustment is continuously arranged in the task sorting result, and each task in the task set participating in the adjustment is at the same level in the directed acyclic graph.

[0193] This implementation manner further considers that in view of the DAG task, the successor task must wait for the predecessor task to complete before it can be started. Therefore, after re-determining the sorting value of the target task, when adjusting the execution order of the target task, it can be restricted that only tasks at the same level can be sorted and adjusted, so as to avoid the situation that the task sorting after adjustment is unreasonable due to the adjustment of tasks at different levels.

[0194] And it can be understood that for tasks that have been completed, their execution order must be before the current target task. Therefore, when the execution order of the target task needs to be adjusted, it is for the unexecuted part of the task sorting result to determine the set of tasks participating in the adjustment, and then based on the sorting value re-determined for the target task, the execution order of the target task is adjusted, and the tasks participating in the adjustment are all tasks in the task set. That is to say, it is equivalent to adjusting the execution order of the target task in the task set.

[0195] And this implementation also takes into account that after establishing a directed acyclic graph including multiple tasks and determining the sorting values of each task in the directed acyclic graph, the sorting values of tasks in the same level are usually relatively close, that is, tasks in the same level will be in relatively close positions in the task sorting result obtained in step S101. And if some tasks in the same level are not close to the tasks in the same level in the task sorting result, for example Figure 5 in the example of, is a schematic diagram of the task sorting result obtained in the embodiment of the present application, Figure 5 the left side is the directed acyclic graph, and the right side is the task sorting result. For Figure 5 in the example of, after determining the sorting values of each task in the directed acyclic graph, in the obtained task sorting result, task 5 is not close to tasks 2 to 4, that is, tasks 2 to 4 are consecutive, but not consecutive with task 5. For such a situation, if the execution order of any task among tasks 2 to 4 is adjusted, task 5 will not participate in the adjustment. For example, if task 2 is the target task and its execution order needs to be adjusted, then since the tasks in the task set participating in the adjustment need to be consecutive tasks in the task sorting result and need to be tasks in the same level in the directed acyclic graph, therefore, the task set participating in the adjustment in this example includes task 2, task 3, and task 4. Such a design takes into account that for tasks in the same level, due to reasons such as large differences in execution cost and communication cost, there may be a situation as shown in Figure 5 where they are not consecutive in the task sorting result. At this time, when adjusting the execution order of the target task, if non-consecutive tasks are also placed in the task set participating in the adjustment, it may cause the target task to cross levels, that is, an abnormal situation where subsequent tasks are placed before their preceding tasks. In summary, this implementation effectively reduces the error probability when adjusting the execution order of the target task by reasonably setting the task set participating in the adjustment.

[0196] Applying the technical solution provided by the embodiments of the present invention, on the basis of arranging tasks in order, the processing units are allocated with the goal of minimizing the overall task execution time. Further, the task order will be dynamically feedback-adjusted to effectively shorten the overall running time of the tasks. Specifically, there are association relationships between tasks, so a directed acyclic graph including multiple tasks needs to be established to reflect such association relationships, and the sorting values of each task in the directed acyclic graph are determined to achieve preliminary sorting, so that all DAG tasks can be executed compactly, reducing the execution time. The DAG tasks described here are also the tasks in the directed acyclic graph. After sorting each task in the directed acyclic graph based on the sorting value and obtaining the task sorting result, each task can be executed in sequence according to the task sorting result. Before executing any task according to the task sorting result, task scheduling is required, that is, determining the most suitable processing unit. And in the solution of this application, when determining the most suitable processing unit for it, not only the task itself is considered, but the processing units are allocated with the goal of minimizing the overall task execution time. Specifically, the scheduling values of each processing unit for the target task are determined. The scheduling value represents the time consumed when all tasks in each task path including the target task in the directed acyclic graph are executed by the corresponding processing unit. Therefore, the processing unit with the lowest scheduling value is the most suitable processing unit for the target task, which can minimize the overall execution time of the DAG tasks.

[0197] Moreover, this application further takes into account the possible situation of resource conflicts. Therefore, after obtaining the scheduling values of each processing unit for the target task, the processing unit with the lowest scheduling value among the processing units that can support the execution of the target task with the available processing resources needs to be used as the optimal processing unit for executing the target task. It can be understood that if the optimal processing unit is the processing unit with the lowest scheduling value among all processing units, then directly use the processing unit with the lowest scheduling value among all processing units as the optimal processing unit and execute the target task. However, when the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units due to resource conflicts, this application solution takes into account the change situation of such processing units, which may affect the scheduling of subsequent tasks. Therefore, the sorting value of the target task will be re-determined, and then for the unexecuted part of the task sorting result, based on the re-determined sorting value of the target task, the execution order of the target task is adjusted to achieve the optimization of task sorting and effectively shorten the overall running time of the tasks. In summary, the solution of this application can effectively implement task scheduling and reduce the overall running time of the tasks.

[0198] Corresponding to the above method and system embodiments, the embodiments of the present invention also provide a task scheduling device, a computer-readable storage medium, and a computer program product, which can be correspondingly referred to with the above text.

[0199] See Figure 6 As shown, the device may include:

[0200] A memory 601 for storing computer programs;

[0201] A processor 602 for executing the computer program to implement the steps of the task scheduling method in any of the above embodiments.

[0202] The computer program product includes computer programs / instructions which, when executed by the processor, implement the steps of the task scheduling method in any of the above embodiments.

[0203] Refer to Figure 7 , a computer program 71 is stored on a computer-readable storage medium 70. When the computer program 71 is executed by the processor, the steps of the task scheduling method in any of the above embodiments are implemented. The computer-readable storage medium 70 mentioned here includes RAM (Random Access Memory), memory, ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), registers, hard disks, removable disks, or any other form of storage medium well-known in the technical field.

[0204] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. For the description of the features in the corresponding embodiments of the task scheduling device, reference can be made to the relevant descriptions in the corresponding embodiments of the task scheduling method, which will not be elaborated here one by one.

[0205] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application. Specific examples are used in this article to elaborate on the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A task scheduling method, characterized in that: include: Establishing a directed acyclic graph including a plurality of tasks, and determining a ranking value of each task in the directed acyclic graph; Sort the tasks in the directed acyclic graph based on the sorting value to obtain a task sorting result, so as to execute the tasks in sequence according to the task sorting result; Before executing any task according to the task sorting result, the task is taken as a target task, and a scheduling value for each processing unit in the processing unit set for the target task is determined; wherein the scheduling value represents the time taken for each task in each task path including the target task in the directed acyclic graph to be completed when the target task is executed by the corresponding processing unit; The processing unit with the lowest scheduling value among the processing units whose available processing resources support the execution of the target task is used as the optimal processing unit for executing the target task, and when the optimal processing unit is not the processing unit with the lowest scheduling value among all the processing units, the ranking value of the target task is re-determined; For the unexecuted task portion in the task sorting result, the execution order of the target tasks is adjusted based on the re-determined sorting values ​​of the target tasks.

2. The task scheduling method according to claim 1, characterized in that: Establishing a directed acyclic graph including a plurality of tasks, and determining a ranking value of each task in the directed acyclic graph, including: Establishing a directed acyclic graph including a plurality of tasks, and determining an initial ranking value of each task in the directed acyclic graph by a preset ranking rule; According to the respective weight coefficients of the tasks, the initial ranking values ​​of the corresponding tasks are adjusted to obtain the ranking values ​​of the tasks in the directed acyclic graph.

3. The task scheduling method according to claim 2, characterized in that: Establishing a directed acyclic graph including a plurality of tasks, and determining an initial sorting value of each task in the directed acyclic graph by a preset sorting rule, including: A directed acyclic graph including multiple tasks is established, and an initial sorting value of each task in the directed acyclic graph is determined through a preset sorting rule. For any task, the initial sorting value of the task is the time taken to complete the execution of all tasks remaining in the task paths including the task in the directed acyclic graph, excluding the predecessor tasks of the task.

4. The task scheduling method according to claim 2, characterized in that: According to the respective weight coefficients of the tasks, the initial ranking values ​​of the corresponding tasks are adjusted to obtain the ranking values ​​of the tasks in the directed acyclic graph, including: For any task, the out-degree of the task is used as the weight coefficient of the task, and the product of the weight coefficient and the initial ranking value of the task is used as the ranking value of the task.

5. The task scheduling method according to claim 1, characterized in that: Before executing any task according to the task sorting result, the task is used as a target task, and a scheduling value of each processing unit in the processing unit set for the target task is determined, including: Before executing any task according to the task sorting result, taking the task as the target task; For any processing unit in the processing unit set, determining the earliest completion time of the target task when the processing unit executes the target task as the forward-looking evaluation time of the processing unit for the target task; Determine, when the processing unit executes the target task, the time taken from the completion of the target task to the completion of the execution of all subsequent tasks of the target task in each task path including the target task in the directed acyclic graph, as the backward evaluation time taken by the processing unit for the target task; The sum of the forward-looking evaluation time and the backward-looking evaluation time is used as the determined scheduling value of the processing unit for the target task.

6. The task scheduling method according to claim 5, characterized in that: For any processing unit in the processing unit set, determining the earliest completion time of the target task when the processing unit executes the target task as the forward evaluation time of the processing unit for the target task, including: For any one of the processing units in the processing unit set, determining the target task execution time when the target task is executed by the processing unit; determining an earliest starting time of the target task when the target task is executed by the processing unit; The target task execution time and the earliest start time of the target task are summed to obtain the earliest completion time of the target task, which is used as the forward-looking evaluation time of the processing unit for the target task.

7. The task scheduling method according to claim 6, characterized in that: Determining the earliest start time of the target task when the processing unit executes the target task includes: Determine each parent task in the directed acyclic graph that points to the target task; For any of the parent tasks, determine the communication time between the parent task and the target task when the processing unit executes the target task, and sum the communication time with the actual completion time of the parent task to obtain the earliest start time of the target task corresponding to the parent task; wherein the actual completion time of the parent task represents: in the directed acyclic graph, in each task path containing the parent task, the time taken for the parent task and each predecessor task of the parent task to be completed; Among the obtained earliest start times of the target task corresponding to each of the parent tasks, a maximum value is selected as the determined earliest start time of the target task when the target task is executed by the processing unit.

8. The task scheduling method according to claim 5, characterized in that: Determining, when the target task is executed by the processing unit, the time taken from the completion of the target task to the completion of the execution of all subsequent tasks of the target task in each task path including the target task in the directed acyclic graph as the backward evaluation time taken by the processing unit for the target task, including: pass In a manner of recursively deducing from the export task in the directed acyclic graph to the target task, a backward evaluation time consumption of the processing unit for the target task is obtained; in, It indicates the export task. represents the kth processing unit, It means that when the kth processing unit executes the export task The task execution time is Indicates export tasks Corresponding to the kth processing unit expenses; represents the i-th round of recursive tasks in the recursive process and It is not an export mission. The task represented It's a task The subtasks of It means the task The overhead set corresponding to the overhead of each processing unit, It means the task With the task The communication between It means that when the task is executed by the kth processing unit The time taken for the task to be executed is max, which means taking the maximum value. It means the task The overhead corresponding to the kth processing unit, Indicates that the kth processing unit is responsible for the task The look-ahead evaluation time of ; k and i are both positive integers.

9. The task scheduling method according to claim 8, characterized in that: When the optimal processing unit is not the processing unit with the lowest scheduling value among all the processing units, re-determining the ranking value of the target task includes: When the optimal processing unit is not the processing unit with the lowest scheduling value among all processing units, re-determining the backward evaluation time consumption of the target task; The re-determined backward evaluation time of the target task is added to the target task execution time when the target task is executed by the optimal processing unit, and the obtained result is adjusted according to the weight coefficient of the target task to obtain the re-determined ranking value of the target task.

10. The task scheduling method according to claim 1, characterized in that: Also includes: Receive directed acyclic graph update instructions; Based on the directed acyclic graph update instruction, the structure of the established directed acyclic graph is updated, and whether the update result meets the set constraints; If not, the structural update of the directed acyclic graph is canceled and a prompt message is output.

11. The task scheduling method according to claim 10, characterized in that: Determine whether the update result meets the set constraints, including: For any task in the update result, when starting from the task and returning to the task after passing through several tasks, it is determined that the update result does not meet the set constraints.

12. The task scheduling method according to claim 1, characterized in that: Also includes: Obtaining the processing resource usage of each processing unit in the processing unit set; When the processing resource usage of any processing unit meets the set reminder rule, prompt information carrying the processing resource usage of the processing unit is output.

13. The task scheduling method according to claim 12, characterized in that: The reminder rules include: The average usage of any processing resource of the processing unit exceeds an upper threshold set for the processing resource, or is lower than a lower threshold set for the processing resource.

14. The task scheduling method according to claim 1, characterized in that: The method further comprises: using a processing unit with the lowest scheduling value among the processing units whose available processing resources support the execution of the target task as the optimal processing unit for executing the target task, including: Arrange the processing units in order from low to high according to the scheduling value to obtain a processing unit selection list for the target task; Determine whether the available processing resources of the xth processing unit in the processing unit selection list of the target task support the execution of the target task; If not, then add 1 to x and return to execute the operation of determining whether the available processing resources of the xth processing unit in the processing unit selection list support the execution of the target task; If yes, the processing unit corresponding to the xth scheduling value is taken as the optimal processing unit; Among them, x is a preset parameter and its initial value is 1.

15. The task scheduling method according to claim 14, characterized in that: Determining whether the available processing resources of the xth processing unit in the processing unit selection list of the target task support the execution of the target task includes: Determine whether the CPU resources, memory resources, and hard disk resources of the xth processing unit in the processing unit selection list of the target task all support the execution of the target task; If both support the execution of the target task, it is determined that the processing unit supports the execution of the target task; otherwise, it is determined that the processing unit does not support the execution of the target task.

16. The task scheduling method according to claim 15, characterized in that: After determining that the available processing resources of the xth processing unit in the processing unit selection list do not support the execution of the target task, the method further includes: Adding the waiting time of the target task to the overhead of the target task corresponding to the xth processing unit, and determining whether the superposition result is less than or equal to the overhead of the target task corresponding to the x+1th processing unit; If yes, the target task is placed in the waiting queue of the xth processing unit, and the xth processing unit is used as the optimal processing unit; If not, then add 1 to x and return to execute the operation of determining whether the available processing resources of the xth processing unit in the processing unit selection list support the execution of the target task; Among them, the target task corresponds to the overhead of the x-th processing unit, which means: when the target task is executed by the x-th processing unit, it means the time taken from the start of execution of the target task to the completion of execution of all subsequent tasks of the target task in the task paths including the target task in the directed acyclic graph.

17. The task scheduling method according to any one of claims 1 to 16, characterized in that: For the unexecuted task part in the task sorting result, adjusting the execution order of the target task based on the re-determined sorting value of the target task, including: For the task part that has not been executed in the task sorting result, determining the task set involved in the adjustment; Based on the re-determined ranking value of the target task, the execution order of the target task is adjusted, and the tasks involved in the adjustment are all tasks in the task set; Among them, the task set participating in the adjustment includes the target task, and each task in the task set participating in the adjustment is a continuously arranged task in the task sorting result, and each task in the task set participating in the adjustment is a task at the same level in the directed acyclic graph.

18. A task scheduling device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the task scheduling method as claimed in any one of claims 1 to 17 when executing the computer program.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the task scheduling method according to any one of claims 1 to 17.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 17 are implemented.

Images