A task scheduling method and device

Abstract

Embodiments of the present application provide a task scheduling method and device, including obtaining a permission value of each task in a plurality of workflows, assigning a priority to the tasks in each workflow in descending order of the permission value, dividing a plurality of target tasks of the same priority in different workflows into the same group, obtaining a level value of each target task in the plurality of target tasks in the same group, scheduling the plurality of target tasks according to the priority of each group, the level value order of the plurality of target tasks of the same priority, and an executable node of each target task, and allocating resources to the tasks of a group with a higher priority first according to the priority grouping, and allocating resources to the tasks with a higher level value in the same priority group first, which considers the priority of the tasks in the same workflow and the level of the tasks in different workflows, can maximize the shortening of task waiting time, speed up the task scheduling time, and meet the fast design requirements of a chip.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, and in particular to a task scheduling method and device. BACKGROUND

[0002] With the rapid development of computer and semiconductor related technologies, chip design and chip manufacturing have also developed rapidly. Before chip manufacturing, chip design is particularly important. Electronic design automation (EDA) as a chip design tool is widely used by chip designers.

[0003] Current EDA tools can complete chip design, simulation, synthesis and verification operations. With the increase in chip design size and design complexity, EDA tools require more and more computing resources. Therefore, the computing resources of current EDA tools have been migrated to high-performance computing clusters to meet the computing power requirements of EDA tools.

[0004] High-performance computing clusters need to process numerous tasks of EDA tools, so multiple tasks need to be scheduled to quickly perform chip design. However, the current task scheduling method has a long task waiting time and a long total task completion time, which cannot meet the rapid design requirements of chips. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a task scheduling method that can shorten the task waiting time, speed up the task scheduling time, shorten the total task completion time, and meet the rapid design requirements of chips.

[0006] To achieve the above purpose, the present application has the following technical solutions:

[0007] The present application provides a task scheduling method, which comprises:

[0008] Obtaining a permission value of each task in a plurality of workflows, the permission value reflecting an executable node of the task, and each workflow comprising at least one task;

[0009] Assigning a priority to the task in each workflow in descending order of the permission value;

[0010] Dividing a plurality of target tasks of the same priority in different workflows into the same group;

[0011] Obtaining a level value of each target task in the plurality of target tasks in the same group;

[0012] Scheduling the plurality of target tasks according to the priority of each group, the level value order of the plurality of target tasks of the same priority, and the executable node of each target task.

[0013] Optionally, the scheduling the plurality of target tasks according to the priority of each group, the order of the level values of the plurality of target tasks in the same priority, and the executable node of each target task comprises:

[0014] determining a first scheduling order of each group in descending order of priority;

[0015] arranging the plurality of target tasks in the same priority in descending order of the level values to obtain a second scheduling order;

[0016] after the plurality of target tasks in the first priority are scheduled according to the second scheduling order, the plurality of target tasks in other priorities are scheduled according to the first scheduling order.

[0017] Optionally, the second scheduling order comprises a plurality of orders, and the scheduling the plurality of target tasks in the first priority according to the second scheduling order comprises:

[0018] determining whether the remaining resources of the executable node corresponding to the target task of the first order meet the execution of the target task of the first order;

[0019] if yes, scheduling the target task of the first order to the executable node corresponding to the target task of the first order;

[0020] if no, continuously determining whether the remaining resources of the executable node corresponding to the target task of the second order meet the execution of the target task of the second order until the target tasks of all scheduling orders in the first priority are determined.

[0021] Optionally, if the executable node corresponding to the target task of the first order is a plurality of executable nodes and the remaining resources of the plurality of executable nodes all meet the execution of the target task of the first order;

[0022] the scheduling the target task of the first order to the executable node corresponding to the target task of the first order comprises:

[0023] obtaining the execution time of the target task of the first order executed by the plurality of executable nodes corresponding to the target task of the first order;

[0024] scheduling the target task of the first order to the executable node with the shortest execution time.

[0025] Optionally, the execution time is obtained by using a time prediction model.

[0026] Optionally, the executable node of each target task is a same node, the node comprises a plurality of resources, and the second scheduling order comprises a plurality of orders.

[0027] the scheduling of the first priority level of the plurality of target tasks in the second scheduling order comprises:

[0028] determining whether the resources of the node satisfy the execution of the target tasks in the first order;

[0029] if yes, allocating resources for the target tasks in the first order;

[0030] if no, continuing to determine whether the remaining resources of the node satisfy the execution of the target tasks in the second order until the scheduling of all target tasks in the first priority level in all scheduling orders is completed.

[0031] Optionally, the permission value of each task is determined according to the execution time of the task in each executable node, the average communication time of the task and other tasks, the permission value of other tasks and the number of executable nodes of the task.

[0032] Optionally, the level value of each target task is determined according to the critical path length CPL of the workflow corresponding to the target task and the number of remaining tasks of the workflow corresponding to the target task.

[0033] Optionally, the method further comprises:

[0034] monitoring the resource information of the executable node of the task in real time.

[0035] Embodiments of the present application also provide a task scheduling device, which comprises:

[0036] a first acquisition unit configured to acquire the permission value of each task in a plurality of workflows, the permission value reflecting the executable node of the task, and each workflow comprising at least one task;

[0037] a first allocation unit configured to allocate priority to the tasks in each workflow according to the descending order of the permission value;

[0038] a second allocation unit configured to divide a plurality of target tasks of the same priority in different workflows into the same group;

[0039] a second acquisition unit configured to acquire the level value of each target task in the plurality of target tasks in the same group;

[0040] a scheduling unit configured to schedule the plurality of target tasks according to the priority of each group, the level value order of the plurality of target tasks in the same priority and the executable node of each target task.

[0041] The embodiment of the present application provides a task scheduling method, comprising: obtaining a permission value of each task in a plurality of work flows, wherein each work flow comprises at least one task, the plurality of tasks belong to different work flows, and the permission value reflects an executable node of the task, that is, the task can only be executed at the executable node; assigning priorities to the tasks in each work flow in descending order of the permission values; dividing a plurality of target tasks with the same priority in different work flows into the same group, that is, performing resource allocation and sequencing on the plurality of tasks in the same work flow; obtaining a level value of each target task in the plurality of target tasks in the same group, that is, performing resource sequencing on the plurality of tasks in different work flows; and scheduling the plurality of target tasks according to the priority of each group, the level value sequence of the plurality of target tasks with the same priority and the executable node of each target task, that is, the plurality of target tasks can be grouped according to the priorities, and subsequently, the tasks in the group with a higher priority can be allocated resources first, and the tasks with a higher level value in the group with the same priority can be allocated resources first. When the task scheduling is performed, the priorities of the tasks in the same work flow are considered, and the levels of the tasks in different work flows are also considered, so that the task waiting time can be maximally shortened, the task scheduling time is accelerated, and when the resources are allocated, the executable node of each task is referred to, so that the node that cannot execute the task is avoided to be allocated to the task, the completion time of all the tasks is further shortened, and the fast design requirement of a chip is met. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0043] Figure 1 A flowchart of a task scheduling method provided by the embodiment of the present application is shown;

[0044] Figure 2 A schematic diagram of a work flow provided by the embodiment of the present application is shown;

[0045] Figure 3 Another schematic diagram of a work flow provided by the embodiment of the present application is shown;

[0046] Figure 4 A schematic diagram of a work flow arrival interval and average queue waiting time provided by the embodiment of the present application is shown;

[0047] Figure 5 A schematic diagram of a work flow arrival interval and average queue waiting time provided by the embodiment of the present application is shown;

[0048] Figure 6 A schematic diagram of a workflow arrival interval and average completion time provided by an embodiment of the application is shown;

[0049] Figure 7 A structural schematic diagram of a task scheduling device provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0051] With the rapid development of computer and semiconductor related technologies, chip design and chip manufacturing have also developed rapidly. The current semiconductor process node has been reduced from 180 nanometers (nm) in 2000 to 3 nm, and the number of transistors on a chip has reached the scale of hundreds of billions from tens of millions. Therefore, before chip manufacturing, chip design is particularly important. Electronic design automation (EDA) as a chip design tool is widely used by chip designers.

[0052] Current EDA tools can complete chip design, simulation, synthesis and verification operations. With the increase of chip design size and the increase of design complexity, EDA tools require more and more computing resources. Therefore, the computing resources of current EDA tools have been migrated to high-performance computing clusters to meet the computing power requirements of EDA tools.

[0053] High-performance computing clusters need to process numerous tasks of EDA tools, so multiple tasks need to be scheduled to quickly perform chip design.

[0054] The current task scheduling method includes a RANK_HYBD method and an online workflow management (OWM) method. The RANK_HYBD method is to put all executable tasks into a preparation queue, and then sort the tasks in the preparation queue according to priorities, if the tasks in the preparation queue all belong to the same workflow, arrange in ascending order, otherwise, arrange in descending order. This method has some problems: if the priority of the new task is higher than the task in the queue, the task in the queue will be delayed, which causes poor fairness, long task waiting time and long total task completion time. The OWM method improves the above method, takes out a task with the highest priority from each workflow into the preparation queue at each scheduling, and sorts the tasks in the preparation queue according to priorities, which is consistent with the RANK_HYBD method. When allocating resources for the task, an adaptive allocation strategy (Adaptive Allocation) is adopted, if there is only one node that meets the resource requirement of the task at present, and the node is the best node (the shortest time to complete the task), the task will be allocated to the node. But the adaptive allocation strategy of the OWM method also has some problems, if the current task is not allocated to the resource, then before the next allocation, the priority of the task in the newly arrived workflow may be higher than that of the task, which will cause the delay of the resource allocation of the current task, resulting in long task waiting time and long total task completion time.

[0055] As can be seen, the current task scheduling method has long task waiting time and long total task completion time, and the current task scheduling method does not consider the problem of license restriction of the task in scheduling, that is, the task can only be executed under the permission of the node, if the node does not allow the task to execute, the resource of the node cannot be allocated to the task, that is, the current task scheduling method cannot meet the rapid design requirements of the chip.

[0056] Based on this, embodiments of this application provide a task scheduling method, including obtaining the permission value of each task in multiple workflows, wherein each workflow includes at least one task, i.e., multiple tasks belong to different workflows, the permission value reflects the executable node of the task, i.e., the task can only be executed at its executable node, assigning priorities to tasks in each workflow in descending order of permission value, grouping multiple target tasks with the same priority in different workflows into the same group, i.e., sorting resource allocation for multiple tasks in the same workflow, obtaining the level value of each target task in the same group, i.e., sorting resources for multiple tasks in different workflows, and assigning priority based on the priority of each group and the multiple tasks with the same priority. The target tasks are scheduled according to their priority levels and the executable nodes of each target task. In other words, they can be grouped according to priority, and resources can be allocated to tasks in higher priority groups first. Within the same priority group, resources can be allocated to tasks with higher priority levels first. In this embodiment, task scheduling considers both the priority of tasks in the same workflow and the priority levels of tasks in different workflows, which can minimize task waiting time and speed up task scheduling. Furthermore, the executable node status of each task is taken into account when allocating resources, which can avoid assigning nodes that cannot execute the task, further shortening the completion time of all tasks and meeting the rapid design requirements of chips.

[0057] To better understand the technical solution and effects of this application, the specific embodiments will be described in detail below with reference to the accompanying drawings.

[0058] refer to Figure 1 The diagram shown is a flowchart of a task scheduling method provided in an embodiment of this application. The method includes the following steps:

[0059] S101, obtain the permission value for each task in multiple workflows.

[0060] In the embodiments of this application, multiple tasks of chip design performed by the EDA tool can belong to multiple workflows, and each workflow includes at least one task; that is, a workflow is a set of tasks. Workflows can be represented using a directed acyclic graph (DAG), denoted as G = {V, E, C, W}, see reference [reference]. Figure 2 As shown in the diagram. Here, V represents the set of tasks in the workflow, E represents the set of directed edges between task nodes, and e... i,j Representing task n i and n j There are directed edges between them, and C represents the set of weights of the directed edges. i,j Represents task n i and n j The average communication time between them, where W represents the set of task execution times, w i,kRepresents task n i The required execution time on node k.

[0061] The critical path in a workflow is the path with the maximum path length from the entry node (Entry) to the exit node (Exit). (See reference) Figure 2 As shown in Table 1, there are 5 paths from the Entry node to the Exit node in the DAG. The path length of each path is shown in Table 1. It can be seen that A→D→G is the critical path of this DAG, so the critical path length of this DAG is 19.

[0062] Table 1. List of all paths in the DAG

[0063] Serial number Path Path length 1 A→C→F 16 2 A→C→G 15 3 A→D→G 19 4 B→D→G 18 5 B→E→G 11

[0064] The critical path length (CPL) of each workflow can be obtained. Specifically, the critical path length of the DAG can be calculated recursively from the Exit node upwards, as shown in the following formula:

[0065]

[0066] in, For task n i Average execution time For task n i With n j The average communication time between them, CPL(n) i ) represents the distance from the Entry node to n i The critical path length of the node. The execution time of the Exit and Entry nodes. Considered as 0, communication time with other tasks It is considered as 0.

[0067] In the embodiments of this application, a License-Based Rank Value in Heterogeneous Environment (LRHE) is obtained for each task. This license value can reflect the executable node of the task, that is, the task can only be executed on its executable node.

[0068] In EDA tools, each task can correspond to a floating license for an executable node, in which case the task can be allocated resources from all nodes for execution. Each task can also correspond to a fixed license for an executable node, in which case the task's usage is restricted, and it can only be assigned to a specific executable node for execution.

[0069] Specifically, the license value for each task is based on the task's execution time w in each executable node.i,k average communication time of tasks and other tasks license value of other tasks LRHE(n j ) and the number of executable nodes of tasks NumS(Pset(n i ).

[0070] As an example, the following formula can be used to obtain:

[0071]

[0072] where Pset(n i ) is the executable node set of task n i , and NumS(Pset(n i )) is the total number of executable nodes of task n i . For EDA tasks using floating licenses, the set is all nodes, and for tasks using fixed licenses or licenses with limited usage range, the set is only nodes corresponding to the license License. i,k Tk(n i is the execution time required by task n i at node k, is the average communication time required by task n i and its child tasks.

[0073] The above formula can be modified by those skilled in the art, and is not limited to the specific means of obtaining the license value of each task.

[0074] S102, prioritize tasks in each workflow in descending order of license value.

[0075] In an embodiment of the present application, after obtaining the license value of each task, each task can be added to a preparation queue to allocate resources to the tasks.

[0076] The tasks in each workflow in the preparation queue can be sorted in descending order of license value, and the multiple tasks can be prioritized according to the sorting. The level value can start from 1 and increase, representing the priority from high to low.

[0077] That is, for tasks in the same workflow, the tasks can be prioritized according to the license value LRHE to determine the order of the tasks.

[0078] S103, divide multiple target tasks of the same priority in different workflows into the same group.

[0079] In the embodiments of the present application, after the tasks in different workflows are sorted according to the license values LRHE, the tasks of the same priority in different workflows can be divided into the same group. That is, the tasks of the same level value are divided into the same group according to the level values of the tasks in different workflows.

[0080] After the tasks of the same level value are divided into the same group, the level values corresponding to each group are also different, that is, each group also has a priority.

[0081] S104, obtaining the level value of each target task in the plurality of target tasks in the same group.

[0082] In the embodiments of the present application, after the tasks of the same priority in different workflows are divided into the same group, the level Rankr value of each target task in the plurality of target tasks in the same group can be obtained, and then the target tasks in the group can be sorted according to the level values to determine the order of resource allocation.

[0083] Specifically, the level Rankr value of each target task can be determined according to the critical path length CPL of the workflow corresponding to the target task and the number of remaining tasks of the workflow corresponding to the target task.

[0084] As an example, the following formula can be used to obtain:

[0085]

[0086] wherein PRT is the percentage of the number of remaining tasks of the workflow corresponding to the target task, and CPL is the critical path length of the workflow corresponding to the target task.

[0087] The above formula can be modified by those skilled in the art, and is not limited to the specific means for obtaining the Rankr value of each target task.

[0088] S105, scheduling the plurality of target tasks according to the priority of each group, the order of the level values of the plurality of target tasks of the same priority, and the executable node of each target task.

[0089] In the embodiments of the present application, after the tasks are grouped according to the license value of each task, the priorities of each group, that is, the level values, are different, the level values of the plurality of target tasks of the same priority are different, and the executable nodes of each target task are different. Subsequently, the plurality of target tasks can be scheduled according to the priority of each group, the order of the level values of the plurality of target tasks of the same priority, and the executable node of each target task.

[0090] Specifically, the first scheduling order of each group can be determined in descending order of priority. Since the tasks of the same priority of different workflows are divided into the same group, the level values of each group are also different. The multiple groups can be sorted in descending order of priority, i.e., in ascending order of level values, to obtain the first scheduling order of the multiple groups.

[0091] Then, the multiple target tasks of the same priority are arranged in descending order of the level values to obtain the second scheduling order. That is, after the tasks of the same priority of different workflows are divided into the same group, the multiple target tasks in the group can continue to be arranged in descending order of the level Rankr values, so as to sort the target tasks of the same level value of different workflows to obtain the second scheduling order of the multiple target tasks.

[0092] After the multiple target tasks of the first priority are scheduled according to the second scheduling order, the multiple target tasks of other priorities continue to be scheduled according to the first scheduling order. That is, the multiple target tasks of the first priority, i.e., the level value of 1, can be allocated resources first. The order of allocating resources is the second scheduling order according to the arrangement of the level values. After the group of the level value of 1 is scheduled, the multiple target tasks of the second priority, i.e., the level value of 2, continue to be allocated resources according to the first scheduling order. After the second scheduling order in the group is continued to be scheduled, the target tasks of other level values continue to be scheduled in turn according to the first scheduling order until the scheduling is completed.

[0093] In the embodiments of the present application, the second scheduling order can include multiple orders, i.e., each group has multiple target tasks, and the level values of the target tasks are different, so each target task has a different order.

[0094] Specifically, the multiple target tasks of the first priority scheduled according to the second scheduling order can be performed in the following two possible implementation manners:

[0095] In the first possible implementation manner, it is determined whether the remaining resources of the executable node corresponding to the target task of the first order meet the execution of the target task of the first order, i.e., whether the remaining resources of the node corresponding to the target task with the highest level value in the level group are sufficient for the execution of the target task. If yes, the target task of the first order is scheduled to the executable node corresponding to the target task of the first order. If no, the target task is not allocated resources in this round, and it is continued to be determined whether the remaining resources of the executable node corresponding to the target task of the second order meet the execution of the target task of the second order, i.e., whether the remaining resources of the node corresponding to the target task with the second highest level value in the level group are sufficient for the execution of the target task. This is continued until the target tasks of all scheduling orders in the first priority are determined according to the order of the level values.

[0096] In practical applications, if the executable nodes corresponding to the target task of the first order are multiple and the residual resources of the multiple executable nodes all satisfy the execution of the target task of the first order, the execution time of the multiple executable nodes executing the target task of the first order can be obtained, that is, the execution time of the multiple executable nodes executing the target task is obtained, and then the target task of the first order is scheduled to the executable node with the shortest execution time, so as to further shorten the execution time between tasks and shorten the completion time of all tasks.

[0097] In the embodiments of the present application, the execution time of the target task on different executable nodes can be obtained in the following three methods:

[0098] The first method: according to the attribute information of the target task, such as the user or application to which the target task belongs, the resource requested by the target task or the requested execution time, etc., the execution time is determined. The requested execution time can be used as the execution time of the executable node. However, this method is mainly applied in the case of no relevant historical data.

[0099] The second method: according to the attribute information of the target task, more relevant historical data is queried, and the number of relevant historical data can be greater than 3. Feature data is generated according to the relevant historical data, which can be the average value or maximum value of the historical execution time, and the feature data is input into a time prediction model to predict the execution time of the target task.

[0100] The third method: if the requested execution time is not included in the attribute information of the target task, the average execution time of similar applications of the application to which the target task belongs is used as the execution time of the target task. Specifically, the execution time of the similar application obtained can be used as the slowest pessimistic estimate T p , that is, the execution time of the fastest node, and other nodes are appropriately lengthened according to the pessimistic estimate T p according to the following formula:

[0101]

[0102] Where S f is the computing power of the fastest node, and S k is the computing power of node k.

[0103] The above introduces three methods of predicting the execution time of the executable node executing the target task, which can assist in task scheduling, so that the task is allocated to the optimal node, that is, the node with the shortest execution time, to further shorten the completion time of all tasks and speed up chip design.

[0104] In a second possible implementation, the executable node of each target task is the same node, which includes multiple resources. It can be determined whether the resources of the node satisfy the execution of the target task in the first order, that is, whether the remaining resources of the node corresponding to the target task with the highest level value in the level group are sufficient for the execution of the target task. If yes, the resources of the node are allocated to the target task in the first order. If no, the resources of the node are not allocated to the target task in the current round, and it is continued to determine whether the remaining resources of the node satisfy the execution of the target task in the second order, until the target tasks in all scheduling orders in the first priority are determined.

[0105] In the embodiments of the present application, the task scheduling method is applied to a computer device, such as a server. The computer device can also detect the resource information of the executable node corresponding to the task and the permission information of the task in real time, so as to timely adjust the task scheduling arrangement. The computer device can also receive the task or workflow submitted by the EDA tool at any time.

[0106] The task scheduling method provided by the embodiments of the present application is described below with a specific example. It is assumed that there is one node in a computer cluster, and the node has 20 resources. It is assumed that there are four workflows A, B, C and D at present, as shown in FIG. 1. After the workflows arrive, the computer device receives multiple tasks in the workflows, and predicts the execution time of the tasks by using the attribute information of the received multiple tasks, so as to obtain the execution time of the tasks to assist the scheduling of the tasks. Figure 3

[0107] The critical path length CPL of each workflow of the four workflows and the permission LHRE value of each task of the multiple tasks can be calculated. When there is an idle resource in the computer cluster, all executable tasks at present are put into a preparation queue. As shown in FIG. 2, A1, A2, B1, B2, C1, C2 and D1 are ready tasks, that is, the executable tasks at present are A1, A2, B1, B2, C1, C2 and D1, and A3, A4, B3-B7, C3-C5 and D2-D5 are unready tasks, that is, the non-executable tasks. As shown in Table 1, the permission LRHE value and the required resource Resource of the current executable tasks are shown. Figure 3

[0108] Table 1: Permission LRHE value and required resource Resource of executable tasks

[0109] Task A1 A2 B1 B2 C1 C2 D1 LRHE 12 9 8 15 13 9 6 Resource 16 4 8 4 20 4 8

[0110] ​​The tasks in each workflow are assigned priorities in descending order of the permitted LRHE values, and multiple target tasks of the same priority in different workflows are divided into the same group, i.e., the tasks belonging to the same workflow are arranged in descending order of their LRHE values, and then the tasks are marked with levels according to the sorting, the task grouping is completed, and the grouping results are that A1, B2, C1, and D1 belong to the first priority and belong to the level_1 group, and tasks A2, B1, and C2 belong to the second priority and belong to the Level_2 group.

[0111] The rank values of each target task in the multiple target tasks in the same group are obtained, and then the multiple target tasks are sorted in descending order of the rank values. In the group of the first priority, i.e., the level_1 group, as shown in Table 2, the Rankr value of task A1 is 0.0833, the Rankr value of task B2 is 0.071, the Rankr value of task C1 is 0.077, and the Rankr value of task D1 is 0.167. The four tasks are arranged in descending order of Rankr, and the order is D1, A1, C1, and B2. In the group of the second priority, i.e., the level_2 group, as shown in Table 3, the Rankr value of task A2 is 0.0833, the Rankr value of task B1 is 0.778, and the Rankr value of task C2 is 0.0769. The tasks are arranged in descending order of Rankr, and the order is A2, B1, and C2.

[0112] Table 2 Resource allocation result of target tasks of the first priority

[0113] Task Rankr Required resource Whether assigned to resource D1 0.167 8 √ A1 0.0833 16 × C1 0.0769 20 × B2 0.0667 4 √

[0114] Next, the target tasks in the level_1 group are allocated resources in the order of priority, i.e., the first scheduling order. There are 20 resources in the cluster, the required resource number of D1 is 8, the resource in the cluster satisfies the resource requirement of task D1, and D1 is allocated to the resource. The required resource numbers of tasks A1 and C1 are 16 and 20 respectively, and after the resource is allocated to D1 according to the second scheduling order, the remaining resources in the cluster are 12, tasks A1 and C1 do not satisfy the resource requirement, and therefore the resource allocation in this round is abandoned. The required resource number of task B2 is 8, and the remaining resources in the cluster satisfy the resource requirement of task B2, and the resource is allocated to B2. The first round of resource allocation is completed, and at this time, there are 8 resources remaining in the cluster.

[0115] Table 3 Resource allocation result of target tasks of the second priority

[0116] Task Rankr Required resource Whether assigned to resource A2 0.0833 4 √ B1 0.778 8 × C2 0.0769 4 √

[0117] The subsequent still follows the first scheduling order, and then resources are allocated to the target tasks in the level_2 group, and 8 resources of the cluster remain. According to the second scheduling order in the level_2 group, resources are first allocated to task A2, and the required resource number of task A2 is 4, and the remaining resources of the cluster meet the requirement of A2, and the resources are allocated to A2. After A2 is allocated resources according to the second scheduling order, the remaining resources of the cluster are 4, the required resource number of task B1 is 8, and the resource requirement of task B1 cannot be met, so the resource allocation of B1 is abandoned in this round, and the required resource number of task C2 is 4, and the remaining resources of the cluster meet the resource requirement of task C2, and the remaining resources are allocated to C2.

[0118] It can be seen that the task scheduling method provided by the embodiment of the application can allocate resources to tasks in a group with a higher priority first in subsequent scheduling, and can allocate resources to tasks with a higher level value in the same group with the same priority. The embodiment of the application considers the priority of tasks in the same workflow and the level of tasks in different workflows when scheduling tasks, can maximize the shortening of task waiting time, speeds up the task scheduling time, and can avoid allocating a node that cannot execute the task to the task when allocating resources, further shortens the completion time of all tasks, and meets the rapid design requirements of a chip.

[0119] Compared with the OWM method, the task scheduling method provided by the embodiment of the application can allocate resources to tasks in each workflow, is more fair, can more fairly schedule workflows, and can reduce the waiting time of workflows.

[0120] The OWM method provided by the embodiment of the application is described below with a specific example: OWM uses upward weight Ranku to schedule, for example:

[0121]

[0122] wherein, is the average computing cost of the task, succ(n j ) is a direct successor task of task ni, is the communication time between tasks ni and nj. In addition, for an exit node, rank u (n exit ) = 0.

[0123] OWM takes out one executable task with the largest Ranku value from each workflow and puts it into the preparation queue each time, and arranges the tasks in the preparation queue in ascending order of Ranku. As shown in Table 4 below, the tasks are assigned resources in the order of D1, A1, C1, B2. The tasks D1 and B2 are assigned resources, and the first round of assignment ends. At this time, there are 8 resources left in the cluster.

[0124] Table 2 OWM first round of assignment results

[0125] Task Ranku Required resource Whether assigned to resource D1 6 8 √ A1 12 16 × C1 13 20 × B2 15 4 √

[0126] In the second round of assignment, OWM still takes out the task with the largest Ranku value from each workflow and puts it into the preparation queue, and arranges the tasks in the preparation queue in descending order of Ranku. The sorting results are shown in Table 5 below, and the resource allocation is started. The cluster resources do not meet the resource requirements of tasks C1 and A1, but meet the resource requirements of task B1, so the resources are allocated to B1. The results are shown in the table below. The second round of scheduling ends.

[0127] Table 3 OWM second round of assignment results

[0128] Task Ranku Required resource Whether assigned to resource B1 8 8 √ A1 12 16 × C1 13 20 ×

[0129] According to the above results, it can be seen that the task scheduling method and the OWM method provided in the embodiments of the present application are consistent in the first round of resource allocation, but show differences in the second round of resource allocation. OWM allocates resources to 2 tasks in workflow B, 1 task in workflow D, and does not allocate resources to tasks in workflows A and C. The task scheduling method provided in the embodiments of the present application allocates resources to tasks in workflows A, B, C and D, that is, the task scheduling method proposed in the present application can more fairly schedule workflows and reduce the waiting time of workflows.

[0130] As an example, the following results are used to demonstrate the effect of the task scheduling method provided in the embodiments of the present application. The cluster configuration is shown in Table 6 below, and there are four different configurations of nodes, and there are also 3 floating licenses (which can be used on any node) and 3 node-bound licenses (which can only be used on fixed nodes) in the cluster. We generated 50 groups of task flows for experiments, each task flow has 20-40 workflows (the number of tasks in each workflow is randomly from 1 to 40, and the execution time is randomly from 100 to 1000), and tested the workflow scheduling results when the workflow arrival interval is from 50 to 300.

[0131] Table 6 Cluster resource configuration table

[0132] Node Speed CPU MEM Fixed license Node0 Fastest 64 128 L1, L3 Node1 Fast 64 128 L2 Node2 Slow 64 128 L1 Node3 Slowest 128 256 L1, L2, L3

[0133] Reference Figure 4 Figure 3 shows a diagram of workflow arrival interval and average queue waiting time provided by the embodiments of the present application. The queue waiting time (t w ) refers to the time from when a workflow is submitted to when the first task of the workflow starts. As shown in the figure, the task scheduling method HEWS provided by the embodiments of the present application has a shorter average queue waiting time (t w ) and higher task scheduling efficiency compared to other methods.

[0134] Reference Figure 5 Figure 4 shows a diagram of workflow arrival interval and average waiting ratio provided by the embodiments of the present application. The average waiting ratio (WR) refers to the ratio of the queue waiting time t w of a workflow to the critical path length CPL of the workflow. The WR is used to measure the delay in the execution of a workflow after the workflow is submitted to the cluster. The lower the WR value, the lower the degree of delay of the workflow. As shown in the figure, the task scheduling method HEWS provided by the embodiments of the present application has a lower WR value, a lower degree of delay of the workflow, and higher task scheduling efficiency compared to other methods.

[0135] Reference Figure 6 Figure 5 shows a diagram of workflow arrival interval and average completion time provided by the embodiments of the present application. The average completion time (t m ) refers to the time from when a workflow is submitted to when all tasks of the workflow are completed. As shown in the figure, the task scheduling method HEWS provided by the embodiments of the present application has a lower average completion time (t m ) and higher task scheduling efficiency compared to other methods.

[0136] The embodiment of the present application provides a task scheduling method, comprising: acquiring a permission value of each task in a plurality of workflows, wherein each workflow comprises at least one task, the plurality of tasks belong to different workflows, and the permission value reflects an executable node of the task, that is, the task can only be executed at the executable node; assigning priorities to the tasks in each workflow in descending order of the permission values; dividing a plurality of target tasks of the same priority in different workflows into the same group, that is, performing resource allocation and sequencing on the plurality of tasks in the same workflow; acquiring a level value of each target task in the plurality of target tasks in the same group, that is, performing resource sequencing on the plurality of tasks in different workflows; and scheduling the plurality of target tasks according to the priority of each group, the level value sequence of the plurality of target tasks of the same priority and the executable node of each target task. That is, the plurality of target tasks can be grouped according to the priorities, and subsequently, the tasks of a group with a higher priority can be allocated resources first, and the tasks with a higher level value in the same priority group can be allocated resources first. When the task scheduling is performed, the priorities of the tasks in the same workflow and the levels of the tasks in different workflows are considered, so that the task waiting time is maximally shortened, the task scheduling time is accelerated, and when the resources are allocated, the executable node of each task is referred to, so that the node that cannot execute the task is avoided to be allocated to the task, the completion time of all the tasks is further shortened, and the fast design requirement of a chip is met.

[0137] Based on the task scheduling method provided in the above embodiment, the embodiment of the present application further provides a task scheduling device, referring to Figure 7 A structural schematic diagram of a task scheduling device provided in the embodiment is shown in the figure, and the device 700 comprises:

[0138] A first acquisition unit 710 is configured to acquire a permission value of each task in a plurality of workflows, wherein the permission value reflects an executable node of the task, and each workflow comprises at least one task;

[0139] A first distribution unit 720 is configured to assign priorities to the tasks in each workflow in descending order of the permission values;

[0140] A second distribution unit 730 is configured to divide a plurality of target tasks of the same priority in different workflows into the same group;

[0141] A second acquisition unit 740 is configured to acquire a level value of each target task in the plurality of target tasks in the same group;

[0142] A scheduling unit 750 is configured to schedule the plurality of target tasks according to the priority of each group, the level value sequence of the plurality of target tasks of the same priority and the executable node of each target task.

[0143] Optionally, the scheduling unit 750 is specifically configured to:

[0144] determining a first scheduling order of each group in descending order of priority;

[0145] arranging multiple target tasks in the same priority in descending order of level values to obtain a second scheduling order;

[0146] after scheduling the multiple target tasks in the first priority according to the second scheduling order, continuing to schedule the multiple target tasks in other priorities according to the first scheduling order.

[0147] Optionally, the second scheduling order includes multiple sequences, and the scheduling unit 750 is specifically configured to:

[0148] determining whether the remaining resources of the executable node corresponding to the target task of the first sequence meet the execution of the target task of the first sequence;

[0149] if the remaining resources meet the execution of the target task of the first sequence, scheduling the target task of the first sequence to the executable node corresponding to the target task of the first sequence;

[0150] if the remaining resources do not meet the execution of the target task of the first sequence, continuing to determine whether the remaining resources of the executable node corresponding to the target task of the second sequence meet the execution of the target task of the second sequence until the target tasks of all scheduling orders in the first priority are determined.

[0151] Optionally, if the executable node corresponding to the target task of the first sequence is multiple and the remaining resources of the multiple executable nodes meet the execution of the target task of the first sequence;

[0152] The scheduling unit 750 is specifically configured to:

[0153] obtaining execution times of the target task of the first sequence executed by the multiple executable nodes corresponding to the target task of the first sequence;

[0154] scheduling the target task of the first sequence to the executable node with the shortest execution time.

[0155] Optionally, the execution time is obtained by using a time prediction model for prediction.

[0156] Optionally, the executable node of each target task is the same node, and the node includes multiple resources; and the second scheduling order includes multiple sequences.

[0157] The scheduling unit 750 is specifically configured to:

[0158] determining whether the resources of the node meet the execution of the target task of the first sequence;

[0159] if the resources meet the execution of the target task of the first sequence, allocating resources for the target task of the first sequence.

[0160] If not, continue to determine whether the remaining resources of the node satisfy the target task execution of the second order until the target tasks of all scheduling orders in the first priority are determined.

[0161] Optionally, the permission value of each task is determined according to the execution time of the task in each executable node, the average communication time of the task and other tasks, the permission value of other tasks and the number of each executable node of the task.

[0162] Optionally, the grade value of each target task is determined according to the critical path length CPL of the workflow corresponding to the target task and the number of remaining tasks of the workflow corresponding to the target task.

[0163] Optionally, the apparatus further comprises:

[0164] The monitoring unit is configured to monitor the resource information of the executable node of the task in real time.

[0165] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the device embodiment is described more simply because it is basically similar to the method embodiment, and the relevant part can be referred to the part of the method embodiment.

[0166] The above only describes the preferred embodiments of the present application. Although the preferred embodiments of the present application are disclosed as above, the present application is not limited to the above. Any person skilled in the art can make many possible changes and modifications to the technical solutions of the present application or make equivalent embodiments with equivalent changes without departing from the scope of the technical solutions of the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the scope of the technical solutions of the present application, shall still fall within the scope of protection of the technical solutions of the present application.

Claims

1. A task scheduling method, characterized by, The method comprises: acquiring a permission value of each task in a plurality of workflows, the permission value reflecting executable nodes of the task, each workflow comprising at least one task; the permission value of each task being determined according to execution time of the task in each executable node, average communication time of the task and other tasks, permission values of other tasks, and the number of executable nodes of the task; assigning priorities to tasks in each workflow in descending order of the permission values; dividing a plurality of target tasks of the same priority in different workflows into the same group; acquiring a rank value of each target task in the plurality of target tasks in the same group; the rank value of each target task being determined according to a critical path length CPL of the workflow corresponding to the target task and the number of remaining tasks of the workflow corresponding to the target task; scheduling the plurality of target tasks according to the priority of each group, the rank value sequence of the plurality of target tasks of the same priority, and the executable nodes of each target task.

2. The method of claim 1, wherein, The scheduling the plurality of target tasks according to the priority of each group, the rank value sequence of the plurality of target tasks of the same priority, and the executable nodes of each target task comprises: determining a first scheduling sequence of each group in descending order of the priority; arranging the plurality of target tasks of the same priority in descending order of the rank value to obtain a second scheduling sequence; after the plurality of target tasks of the first priority are scheduled according to the second scheduling sequence, continuing to schedule the plurality of target tasks of other priorities according to the first scheduling sequence.

3. The method of claim 2, wherein, The second scheduling sequence comprises a plurality of sequences, and the scheduling the plurality of target tasks of the first priority according to the second scheduling sequence comprises: judging whether the remaining resources of the executable nodes corresponding to the target task of the first sequence satisfy execution of the target task of the first sequence; if yes, scheduling the target task of the first sequence to the executable nodes corresponding to the target task of the first sequence; if no, continuing to judge whether the remaining resources of the executable nodes corresponding to the target task of the second sequence satisfy execution of the target task of the second sequence until all target tasks of the scheduling sequences of the first priority are judged.

4. The method of claim 3, wherein, if the executable nodes corresponding to the target task of the first sequence are a plurality of nodes and the remaining resources of the plurality of executable nodes satisfy execution of the target task of the first sequence; the scheduling the target task of the first sequence to the executable nodes corresponding to the target task of the first sequence comprises: acquiring execution time of the target task of the first sequence by the plurality of executable nodes corresponding to the target task of the first sequence; scheduling the target task of the first sequence to the executable node with the shortest execution time.

5. The method of claim 4, wherein, The execution time is obtained by using a time prediction model.

6. The method of claim 2, wherein, The executable nodes of each target task are the same node, the node comprising a plurality of resources; and the second scheduling sequence comprises a plurality of sequences. The scheduling the plurality of target tasks of the first priority according to the second scheduling sequence comprises: judging whether the resources of the node satisfy execution of the target task of the first sequence; If the condition is satisfied, resources are allocated to the target task of the first order; If the condition is not satisfied, it is determined whether the remaining resources of the node satisfy the target task execution of a second order until the target tasks of all scheduling orders in the first priority are determined.

7. The method according to any one of claims 3 to 6, characterized in that, The method further comprises: Real-time monitoring of resource information of the executable node of the task.

8. A task scheduling apparatus characterized by comprising: The device comprises: A first obtaining unit is configured to obtain a permission value of each task in a plurality of workflows, wherein the permission value reflects an executable node of the task, each workflow comprises at least one task, and the permission value of each task is determined according to execution time of the task in each executable node, average communication time of the task and other tasks, permission values of other tasks, and the number of each executable node of the task; A first allocating unit is configured to allocate a priority to a task in each workflow in descending order of the permission value; A second allocating unit is configured to divide a plurality of target tasks of the same priority in different workflows into the same group; A second obtaining unit is configured to obtain a level value of each target task in the plurality of target tasks in the same group, wherein the level value of each target task is determined according to a critical path length CPL of a workflow corresponding to the target task and the number of remaining tasks of the workflow corresponding to the target task; A scheduling unit is configured to schedule the plurality of target tasks according to the priority of each group, the level value order of the plurality of target tasks of the same priority, and the executable node of each target task.

Images