Task Scheduling Method, Electronic Device, Storage Medium and Program Product

Abstract

The present application discloses a task scheduling method, an electronic device, a storage medium, and a program product, relating to the field of computer technologies. When obtaining a target task with a dependency relationship, when selecting a computing node to execute the current task, the execution duration of the entire current task link to which the current task belongs, that is, the predicted duration, can be considered. In this way, a computing node whose predicted duration meets a preset condition can be selected as the target node for executing the current task, and the current task is scheduled to the target node for execution, so as to comprehensively and globally improve the execution efficiency of the current task and its subsequent tasks. It solves the technical problem that unreasonable task scheduling easily leads to task progress lag, and achieves the technical effect of reasonably planning the computing nodes for realizing tasks to improve task processing efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a task scheduling method, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Cloud computing technology is a form of computing in which computing resources are provided to users in the form of services through a network. Through cloud computing, it is convenient for users to obtain computing resources and services, and appropriate computing nodes can be selected to implement one or more tasks sent by the users. Among them, when implementing multiple tasks of users, a certain scheduling strategy is required to allocate reasonable computing nodes to each task. However, due to the unreasonable current task scheduling strategy, there are easily problems such as unsatisfactory task processing efficiency and even lag in task progress. Summary of the Invention

[0003] This application provides a task scheduling method, an electronic device, a computer-readable storage medium, and a computer program product to at least solve the problem in the related art that unreasonable task scheduling easily leads to lag in task progress.

[0004] This application provides a task scheduling method, which includes: obtaining a set of to-be-executed tasks of a preset object; where the set of to-be-executed tasks includes multiple target tasks with dependencies; respectively evaluating the predicted duration of each computing node in the computing cluster for executing the current task link; where the current task link includes the current task and successor tasks, and the successor tasks represent other target tasks that depend on the current task; selecting a computing node whose predicted duration meets the preset conditions as the target node, and scheduling the current task to the target node to make the target node execute the current task.

[0005] This application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above task scheduling methods when executing the computer program.

[0006] This application also provides a computer-readable storage medium, in which a computer program is stored, and where the computer program implements the steps of any of the above task scheduling methods when executed by a processor.

[0007] This application also provides a computer program product, including a computer program, and the computer program implements the steps of any of the above task scheduling methods when executed by a processor.

[0008] Through the present application, when obtaining target tasks with dependencies, when selecting a computing node to execute the current task, the execution duration of the entire current task link to which the current task belongs, that is, the predicted duration, can be considered. Thus, a computing node whose predicted duration meets the preset conditions can be selected as the target node for executing the current task, and the current task can be scheduled to the target node and executed, so as to comprehensively consider the execution efficiency of the current task and its subsequent tasks globally, and reduce the scheduling process of continuously migrating each target task in the current task link between computing nodes. Therefore, the technical problem that unreasonable task scheduling easily leads to task progress lag can be solved, and the technical effect of reasonably planning the computing nodes for task implementation to improve task processing efficiency can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0010] Figure 1 It is a schematic structural diagram of an embodiment of the task scheduling system of the present application;

[0011] Figure 2 It is a schematic flowchart of an embodiment of the task scheduling method of the present application;

[0012] Figure 3 It is a schematic flowchart of another embodiment of the task scheduling method of the present application;

[0013] Figure 4 It is a schematic structural diagram of an embodiment of the dependency topology of the present application;

[0014] Figure 5 It is a schematic structural diagram of an embodiment of the computing node sequence of the present application;

[0015] Figure 6 It is a schematic structural diagram of an embodiment of the scheduling computing node sequence of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0017] It should be noted that in the description of this application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0018] In order to enable those skilled in the art of this technical field to better understand the solution of this application, the following further describes this application in detail with reference to the accompanying drawings and specific embodiments.

[0019] First, in combination with the specific application environment architecture or specific hardware architecture on which the execution of the task scheduling method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0020] Specifically, please refer to Figure 1 , Figure 1 which is a schematic structural diagram of an embodiment of the task scheduling system of this application.

[0021] In one embodiment, the task scheduling system can be applied to cloud computing. Cloud computing is a distributed computing that can decompose a huge data computing and processing program into multiple small programs through communication means such as a network, and then process and analyze these small programs through a computing cluster including multiple computing nodes to obtain results, and can return the results to a preset object that issues a request.

[0022] Specifically, the task scheduling system can include a task scheduling device and a computing cluster. The task scheduling device can be set in the computing cluster or can be relatively independent of the computing cluster, and this is not limited herein. Figure 1 shows an implementation manner in which the task scheduling device is set in the computing cluster as an example.

[0023] As the name implies, the task scheduling device is used to schedule and execute the target task in the set of tasks to be executed fed back by the preset object, that is, it can schedule the computing nodes for executing the target task.

[0024] The preset object is the external object of the service provided in this embodiment, and can be considered equivalent to a user. In this embodiment, the task scheduling device and the task scheduling method described later can perform task scheduling on the set of tasks to be executed of a single preset object, that is, the number of preset objects is one. It can be understood that this embodiment is for task scheduling and task execution of the set of tasks to be executed of one user.

[0025] The set of tasks to be executed can be regarded as the request sent by the user. In this embodiment, the set of tasks to be executed may include multiple target tasks with dependencies, and the target tasks can be considered as services, computations, etc. that need to be implemented. Among them, the set of tasks to be executed can represent multiple target tasks decomposed from a request sent by a preset object; or it can represent a set of multiple target tasks with dependencies sent by a preset object, and no strict limitation is made here.

[0026] A computing node represents a computer and its supporting devices in the field of computer technology, and can be applied as a storage and / or computing unit in a distributed computing architecture. For example, a computing node can be a server, etc. A server is a specific IT device that provides computing power and runs software applications in a network environment, and it can provide computing or application services for other client machines (such as terminal devices like personal computers, smartphones, etc.) in the network. Generally speaking, a server can have the ability to undertake response service requests, undertake services, and guarantee services.

[0027] A computing cluster includes multiple interconnected computing nodes. That is, multiple computing nodes form a computing cluster, and the multiple computing nodes are connected to each other to achieve communication and cooperation between the computing nodes. As exemplified by the computing node being a server in the previous text, the computing cluster can be a server cluster, and the server cluster can be composed of multiple servers interconnected through an input / output system.

[0028] Specifically, in this embodiment, the macroscopic task scheduling process is that the computing cluster can receive the set of tasks to be executed from a preset object. The task scheduling device can schedule each target task with dependencies in the tasks to be executed, that is, schedule the execution subject and / or execution order, etc. of each target task, select a computing node suitable for it as the target node for each target task, and make the target node execute the target task.

[0029] The following gives an example of the process of the task scheduling device in this application for scheduling. That is, the embodiment of this application specifically provides a task scheduling method. The following describes the method in detail in combination with the execution flow of the task scheduling method.

[0030] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of an embodiment of the task scheduling method of this application.

[0031] S101: Obtain the set of tasks to be executed of a preset object; among them, the set of tasks to be executed includes multiple target tasks with dependencies.

[0032] In this embodiment, as exemplified in the foregoing, the target tasks in the task set to be executed can belong to a preset object. In other words, the task scheduling method in this embodiment can be applied to the task set to be executed for a preset object, so as to deeply perceive the dependency relationships between the target tasks while achieving the target tasks that meet the requirements of the preset object, and to reasonably schedule the task process and the task execution process.

[0033] Among them, the dependency relationship can represent a direct or indirect dependency between two target tasks; and / or, two target tasks can have a common directly or indirectly dependent target task; and / or, two target tasks can have a target task that directly or indirectly depends on both of them at the same time.

[0034] S102: Evaluate the predicted execution duration of each computing node in the computing cluster for the current task link respectively; where the current task link includes the current task and the successor tasks, and the successor tasks represent other target tasks that depend on the current task.

[0035] In this embodiment, the task set to be executed of the preset object can be sent to the computing cluster. The computing cluster includes multiple computing nodes, and the computing nodes for executing each target task are scheduled through task scheduling, so that the computing nodes execute the corresponding computing nodes in the task set to be executed.

[0036] The current task can be understood as the target task to be scheduled and executed currently. At the same time, as described in the foregoing, there are certain dependency relationships between the target tasks in the task set to be executed. When intending to complete all the target tasks in the task set to be executed, the current task can have successor tasks, and the successor tasks represent other target tasks that depend on the current task. Generally speaking, it can be considered that the successor tasks are the target tasks that need to be executed after the current task is completed. It can be that the successor tasks require the output of the current task, and / or, it can be that the successor tasks can improve the execution reliability when executed after the current task is completed, etc., which are not limited here.

[0037] That is to say, in this embodiment, when scheduling the current task, the execution duration of the current task link on each computing node is considered globally. In this embodiment, it is expected that the current task and the successor tasks of the current task link can be realized on the same computing node, which can improve the situation that the target tasks of the same task link are scattered on multiple computing nodes, and is beneficial to improving the consistency of the scheduling and execution of the current task link. Moreover, in this embodiment, the predicted execution duration of the computing node for executing the current task link is used as the measurement dimension for selecting the computing node, so as to facilitate ensuring the scheduling and execution efficiency of the current task and the current task link.

[0038] S103: Select the computing nodes with a prediction duration meeting the preset conditions as target nodes, schedule the current task to the target nodes, and let the target nodes execute the current task.

[0039] In this embodiment, the target nodes refer to the computing nodes selected to execute the current task. The computing nodes serving as target nodes can be selected based on the prediction duration. After the target nodes are selected, the current task can be scheduled to the target nodes for the target nodes to execute the current task.

[0040] Among them, the preset conditions can refer to selecting the shortest prediction duration; or, it can be to select the prediction duration between the preset duration thresholds and the remaining computing resources are sufficient, etc., which are not limited here.

[0041] Thus, in this embodiment, since when obtaining the target tasks with dependencies, when selecting the computing nodes to execute the current task, the execution duration of the entire current task link to which the current task belongs, that is, the prediction duration, can be considered. In this way, the computing nodes with a prediction duration meeting the preset conditions can be selected as the target nodes to execute the current task, the current task can be scheduled to the target nodes and executed, so as to comprehensively consider the execution efficiency of the current task and its subsequent tasks globally, and the scheduling process of continuously migrating the target tasks in the current task link between computing nodes can be reduced. Therefore, the technical problem that unreasonable task scheduling easily leads to task progress lag can be solved, and the technical effect of reasonably planning the computing nodes for task implementation to improve task processing efficiency can be achieved.

[0042] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of another embodiment of the task scheduling method of this application.

[0043] S201: Obtain the set of tasks to be executed of a preset object.

[0044] In this embodiment, as elaborated above, the set of tasks to be executed can include multiple target tasks with dependencies, and the multiple target tasks can belong to the same preset object.

[0045] S202: Perform resource abstraction on the computing cluster.

[0046] In this embodiment, resource abstraction processing can be performed on the computing nodes in the computing cluster to facilitate subsequent rapid identification of the computing nodes capable of executing the target tasks.

[0047] Specifically, the computing nodes can be abstractly expressed as processor = [P, available CPU capacity, available memory capacity, available number of GPU cores, available GPU video memory capacity]. Among them, P can be expressed as P = {p k |p k{indicating the k-th processor in the system}, the available CPU capacity, available memory capacity, available number of GPU cores, and available GPU video memory capacity are the capacities available for tasks on the computing node, and their values can be updated in real time through the system monitoring service.

[0048] S203: Construct the dependency topology between target tasks.

[0049] In this embodiment, based on the dependency relationships between target tasks in the task set to be executed, a dependency topology such as a directed acyclic graph can be constructed between target tasks. In this way, relevant data can be obtained through the dependency topology, improving the efficiency of information acquisition during task scheduling, and also facilitating the improvement of the integrity of information acquisition. It can reduce omissions such as backward tasks and forward tasks, omissions of task links, and misjudgments of starting tasks and ending tasks, while improving the clarity of information acquisition and facilitating the improvement of the reliability of task scheduling. As Figure 4 exemplified in Figure 4 is a schematic structural diagram of an embodiment of the dependency topology of the present application.

[0050] For example, in this embodiment, the dependency topology can be described by a quadruple, that is, DAG = [V, E, W, C]. Among them, DAG represents the dependency topology.

[0051] V represents the task set to be executed. Specifically, V can represent a set of multiple target tasks, and each target task can be respectively represented as t1, t2,..., t n , where n is a positive integer. The task request resource information of the target task can be included in t i , where i is a positive integer between 1 and n. For example, t i can also be represented by a quadruple, that is, T = [CPU request amount, memory request amount, number of GPU cores request amount, GPU video memory request amount]. The CPU (Central Processing Unit) can be the operation and control core of a computer system and is the final execution unit for information processing and program operation. The GPU (Graphics Processing Unit) is a coprocessor used to process image and graphics operations.

[0052] E is used to express the dependency relationship between target tasks, that is, E = {e i,j |e i,j represents the communication constraint relationship from t i to t j}; where 1 ≤ i < j ≤ n, that is, there is a sequential execution relationship between the i-th target task (t i ) and the j-th target task (t j ), and t j is before ti Execute after the execution is completed, and if t j Has multiple parent nodes (i.e., the target tasks it depends on), and can wait for all of them to be t j After the target tasks of the parent nodes are executed and the result data of the parent nodes is transmitted and arrives, start to execute t j .

[0053] W represents the estimated execution time overhead of the target task on the computing node, that is, the estimated time. Specifically, W = {w i,k |w i,k represents the estimated time of the i-th target task on the k-th computing node}. Before the target task actually starts to execute, its approximate execution time on the k-th computing node can be estimated, and this estimated time can be the one estimated when the resources required by the target task can be satisfied.

[0054] C represents the estimated information transfer time between computing nodes, that is, the communication time. Specifically, C = {c i,j |c i,j represents the pre-estimated information transfer time from the target task t i to the target task t j}. Further, C = {c i,j,k |c i,j,k represents the pre-estimated information transfer time from the target task t i to the target task t j on the k-th computing node}.

[0055] S204: Obtain the end task and the start task in the set of tasks to be executed.

[0056] In this embodiment, the start task of the target task in the set of tasks to be executed can be obtained, and the end task of the target task in the set of tasks to be executed can be obtained. Among them, the start task and the end task are also target tasks. Optionally, the dependency topology can be queried to obtain the end task and the start task.

[0057] Further, the forward tasks and backward tasks of each target task can also be obtained to form a forward task set and a backward task set of the target task, so as to quickly obtain relevant data in subsequent processes such as link cost evaluation.

[0058] Specifically, the forward task set of the target task can be expressed as pred(t i ). Further, the forward tasks included in pred(t i ) can be direct predecessor target tasks, that is, the target task t iThe adjacent (i.e., having a direct dependency) forward target task. If so, the computational amount of the link cost in subsequent steps can be significantly reduced. And the forward task set of the starting task can be an empty set, that is to say, pred(t i ) can also be identified. When pred(t i ) is an empty set, it is determined that t i is the starting task, which is equivalent to the entry task.

[0059] The backward task set of the target task can be expressed as succ(t i ). Further, the backward tasks included in succ(t i ) can be direct successor target tasks, that is, the adjacent (i.e., having a direct dependency) backward target tasks of the target task t i . If so, the computational amount of the link cost in subsequent steps can be significantly reduced. And the backward task set of the starting task can be an empty set, that is to say, succ(t i ) can also be identified. When succ(t i ) is an empty set, it is determined that t i is the end task, which is equivalent to the exit task.

[0060] S205: Evaluate the link cost of the target task.

[0061] In this embodiment, the link costs of each computing node for executing the target task can be independently evaluated. Calculate the average value of the link costs of each computing node as the sorting factor of the target task.

[0062] Optionally, the backward time overhead of the target task can be used as the predicted link overhead, that is, the backward time overhead of the computing node for executing the target task can be used as the link cost.

[0063] Specifically, the estimated time for the computing node to execute the target task can be obtained. Obtain the backward time overhead of the predicted backward task; where the backward task represents other target tasks that depend on the target task; obtain the pre-evaluated communication time from the target task to the backward task as the third time overhead; superimpose the estimated time, the backward time overhead of the backward task, and the third time overhead to obtain the backward time overhead of the target task, and use the backward time overhead of the target task as the link cost. If so, in this embodiment, through reasonable selection of the link cost, the backward time overhead can be used as the link cost and can also be used for the selection of subsequent target computing nodes, thereby improving the scheduling efficiency of each target task in the task set to be executed as the current task. The selection of the target node will be elaborated in detail later.

[0064] The following is an example of the calculation formula for the backward time overhead in this embodiment:

[0065] Formula 1-1

[0066] where t i represents the i-th target task, and i is a positive integer; p k represents the k-th computing node, and k is a positive integer; rank u (t i , p k ) represents the link cost for the k-th computing node to execute the i-th target task; t j represents the j-th target task, and the j-th target task depends on the i-th target task; succ(t i ) represents the set of backward tasks of the i-th target task, including other target tasks that depend on the i-th target task; rank u (t j , p k ) represents the link cost for the k-th computing node to execute the j-th target task, that is, the predicted backward time overhead of the backward task t j ; c i,j represents the communication time from the i-th target task to the j-th target task, that is, the third time overhead; w i,k represents the estimated time for the k-th computing node to execute the i-th target task; max represents taking the maximum value, since there may be multiple backward tasks for the target task.

[0067] Different from the successor tasks mentioned in the previous text, successor tasks can represent other target tasks that depend on the current task. In this embodiment, backward tasks can represent target tasks that depend on the target task being evaluated during the evaluation of the target task. When the target task is the current task, the backward tasks that depend on it can be used as successor tasks. There may be other considerations when specifically selecting them as successor tasks, which will not be elaborated here.

[0068] S206: Assign a sorting factor to the target task that is positively correlated with its link cost.

[0069] In this embodiment, the backward time overhead for each computing node to execute the target task is calculated respectively as the link cost for each computing node to execute the target task. Considering that the number of computing nodes that can implement the target task may be multiple, the link costs can be processed to obtain the sorting factor. For example, the link costs can be weighted and fused, fitted, averaged, etc. The following gives an example of the formula for averaging the link costs for each computing node to execute the target task:

[0070] Formula 1-2

[0071] where hrank u (ti ) represents the sorting factor of the i-th target task; rank u (t i , p k ) represents the link cost for the k-th computing node to execute the i-th target task; |P| represents the number of computing nodes in the computing cluster.

[0072] Thus, this embodiment can ensure the orderly and efficient parallel execution of target tasks in the processor system, and can also make multiple target tasks to be executed have better compactness, which is beneficial to making the set of tasks to be executed have the characteristic of approaching the minimum execution time while reducing the resource fragmentation of the computing cluster, thereby improving the resource utilization rate of the computing cluster.

[0073] Furthermore, to further improve the generality of task scheduling in this embodiment and improve task scheduling efficiency. The processor types can include homogeneous processors and heterogeneous processors. That is, the processor types of the computing cluster can be obtained. When the processor type of the computing cluster is a heterogeneous processor, it can be evaluated according to the sorting factor methods exemplified by equations 1-1 and 1-2. When the processor type of the computing cluster is a homogeneous processor, the average execution time of each computing node in the computing cluster on each processor can be evaluated, and the maximum value of the backward time overhead of the backward task depending on the target task and the average communication duration between the target task and the backward task is evaluated, and the sum value of this maximum value and the average execution time is used as the link cost of the target task, so as to ensure the orderly and efficient parallel execution of target tasks in the processor system, and can also make multiple target tasks to be executed have better compactness, which is beneficial to making the set of tasks to be executed have the characteristic of approaching the minimum execution time while reducing the resource fragmentation of the computing cluster, thereby improving the resource utilization rate of the computing cluster. At the same time, through the average execution time of the computing nodes, it is also possible to reduce the calculation of the execution time of each computing node separately, which can significantly improve the scheduling efficiency.

[0074] It can be evaluated according to the following sorting factor calculation method.

[0075] Equation 2-1

[0076] Among them, rank u (t i ) represents the link cost of the i-th target task under the homogeneous processor type; represents the average execution time for each computing node to process the i-th target task; rank u (t j ) represents the link cost of the j-th target task under the homogeneous processor type; represents the average communication duration from the i-th target task to the j-th target task. rank u (t i ) can be used as a sorting factor.

[0077] The following gives an example of the calculation formula for and :

[0078] Equation 2-2

[0079] Equation 2-3

[0080] where P represents the computing cluster and p represents the computing node; ct i,p represents the execution time of a computing node for executing the i-th target task; |p| represents the number of computing nodes for the cumulative execution time in the denominator; represents the communication time from the computing node executing the i-th target task to the computing node executing the j-th target task, that is, p i represents the computing node executing the i-th target task, and p j represents the computing node executing the j-th target task; |P|*|P| represents the number of communication paths between the computing nodes executing the i-th target task and the computing nodes executing the j-th target task in the numerator of Equation 2-3.

[0081] S207: Sort the execution priorities of the target tasks based on the sorting factor.

[0082] In this embodiment, the sorting factors of the target tasks in the to-be-executed task set can be compared to sort the execution priorities of the target tasks, and the execution priorities of the target tasks are controlled to be positively correlated with the sorting factors. Generally speaking, in this embodiment, the target tasks with higher execution priorities can be scheduled and executed earlier than the target tasks with lower priorities. In this implementation, it is expected to preferentially schedule and execute the target tasks and their task links that consume more link costs, so that they can start scheduling and execution earlier, move forward the time when they are completed, and advance the completion time of the to-be-executed task set.

[0083] Further, after sorting the execution priorities of target tasks, it is also possible to review the execution order of the target tasks after sorting the execution priorities. As the name implies, target tasks with higher execution priorities can be scheduled and executed earlier. Therefore, before scheduling and executing the current task, the positions of the current task and other target tasks that have not been scheduled and executed can be located in the dependency topology. Based on the located positions, it can be confirmed whether the current task depends on other target tasks that have not been scheduled, so as to review whether the execution priorities of the target tasks are reliable, which is beneficial to reducing situations such as skipping levels and ensuring the reliability of the scheduling and execution of the task set to be executed. When it is confirmed that the current task does not depend on other unscheduled target tasks, the current task can be continuously scheduled and executed. When it is confirmed that the current task depends on other unscheduled target tasks, the execution priority of the current task is lowered to be lower than the execution priority of the target tasks it depends on, and the current task is re-obtained based on the execution priority.

[0084] S208: Obtain the current task based on the execution priority.

[0085] In this embodiment, the current task can be sequentially selected in the order of descending execution priority, so as to be able to advance the execution completion time of the task set to be executed.

[0086] S209: Select the current task link of the current task based on the link cost.

[0087] In this embodiment, specifically, the task links from the current task to the end task in the dependency topology can be traversed. And the link costs of the task links from the current task to the end task are respectively evaluated. To select the task link with a link cost higher than other link costs as the current task link. Generally speaking, in this embodiment, the task link with the highest link cost can be used as the current task link, so as to preferentially allocate computing nodes to the current task link with the highest link cost, which is beneficial to improving the scheduling and execution start time and efficiency of the current task link. After completing the scheduling of the current task, that is, the current task link, the scheduling of other task links with relatively lower link costs can also be carried out. Furthermore, it is possible to advance the scheduling of the current task link with the highest link cost, which is beneficial to globally advancing the execution completion time of the task set to be executed.

[0088] Optionally, the link cost may include: the path length between the current task and the end task; where the path length is in a proportional relationship with the number of target tasks between the current task and the end task. And / or, the predicted link overhead for executing the current task to the end task based on the task link.

[0089] For example, the link cost can be represented solely by the path length. Usually, the path length is also positively correlated with the link cost of the task link. Representing the link cost by the path length is beneficial to reducing the operation efficiency of the link cost. Representing the link cost by the predicted link overhead is beneficial to improving the accuracy of the link cost, and can improve the rationality of the planned task scheduling strategy, thereby being beneficial to improving the execution efficiency of the task set to be executed.

[0090] Alternatively, the path length and the predicted link overhead can be taken into account to comprehensively consider the link cost. The path length and the predicted link overhead can be superimposed, weighted and fused, fitted, etc., which are not limited herein.

[0091] Optionally, the predicted link overhead includes the predicted backward time overhead of each computing node for executing the task link.

[0092] As the link cost calculated in the previous formula 1-1, take the backward time overhead of the i-th target node calculated by rank u (t i , p k ) as the link cost.

[0093] S210: Evaluate the predicted duration of each computing node in the computing cluster for executing the current task link respectively.

[0094] In this embodiment, the current task link may include the current task and the successor task, and the successor task represents other target tasks that depend on the current task.

[0095] Specifically, the forward time overhead from the start task to the current task can be obtained; the predicted backward time overhead of each computing node is respectively superimposed with the forward time overhead to obtain the predicted duration of each computing node, so as to globally consider the time overhead of the computing node for executing the current task link.

[0096] Furthermore, the first time overhead for executing the start task to the forward task can be obtained; wherein, the forward task represents the adjacent forward target task of the current task; the communication time from the forward task to the current task is obtained as the second time overhead; the sum of the first time overhead and the second time overhead is used as the forward time overhead. In this way, the real forward task execution time can be combined to participate in the scheduling process of the current task, which can improve the reliability and accuracy of the current task scheduling process. Optionally, the first time overhead for actually executing the forward task can also be compared with the estimated time overhead, so as to update the estimated time for executing the target task using the difference between the two, thereby being able to improve the accuracy of the estimated time during the task scheduling and execution process, so as to improve the accuracy of the link cost and the predicted duration, and further improve the reliability of the task scheduling in this embodiment.

[0097] Formula 1-3

[0098] Wherein, EST(t i , p k ) represents the forward time overhead for executing up to the current task; t h represents the h-th target task, the i-th target task depends on the h-th target task, that is, the h-th target task is the forward task of the i-th target task; pred(t i ) represents the set of forward tasks of the i-th target task; AFT(t h ) represents the first time overhead for executing from the starting task to the completion of the h-th target task; c h,i represents the communication time from the h-th target task to the i-th target task, that is, the second time overhead.

[0099] Wherein, EST(t i , p k ) can also be regarded as the earliest start time of the i-th target task. That is, when calculating the link cost from the starting task to it on the k-th processor, the maximum path or overhead among the previous nodes can be taken. When multiple target tasks are executed in parallel, the earliest start time of the executed target task is determined by the one with the longest execution time in the same batch of target tasks, which can improve the reliability of the estimated earliest start time of the target task.

[0100] S211: Select a computing node whose prediction duration meets the preset conditions as the target node.

[0101] In this embodiment, compare the prediction durations of the current task links executed by each computing node. Select a computing node whose prediction duration is less than the prediction durations of other computing nodes as the target node. It can be equivalent to scheduling the current task to the computing node considered optimal in this embodiment, that is, the time for the current task link to complete execution at the target node can be earlier than the execution completion time at other computing nodes. Thus, by reasonably allocating computing nodes for the current task, the scheduling and execution efficiency of the current task link can be improved. The following is an example of the formula for calculating the prediction duration in this embodiment:

[0102] Formula 1-4

[0103] Wherein, EFT(t i , p k ) represents the time when the i-th target task is completed; w i,k represents the estimated time for the k-th computing node to execute the i-th target task, and k is a positive integer, and can be calculated separately for each computing node in the computing cluster.

[0104] Equation 1-5

[0105] where, LDFE(t i , p k ) represents the backward time overhead of the backward task of the i-th target task executed by the k-th computing node.

[0106] Equation 1-6

[0107] where, p ps0 represents the predicted duration of the computing node selected as ps0, and ps0 represents the computing node used as the target node.

[0108] As Figure 5 illustrated by the example shown in Figure 5 is the structural schematic diagram of an embodiment of the computing node sequence of the present application. Figure 5 In

[0109] S212: Schedule the current task to the target node.

[0110] In this embodiment, the current task can be scheduled to the target node, expecting the target node to execute the current task.

[0111] Furthermore, obtain the remaining computing resources of the target node. Determine whether the remaining computing resources support the execution of the current task, which is beneficial to ensuring the stability of the computing cluster and at the same time can also take into account the reliable execution of the current task.

[0112] In response to whether the remaining computing resources support the execution of the current task, instruct the target node to execute the current task.

[0113] In response to the remaining computing resources not supporting the execution of the current task, evaluate whether to select another computing node as the new target node. When it is determined to select another computing node as the new target node, schedule the current task to the new target node for execution. When it is determined not to select another computing node as the new target node, wait for the computing resources of the target node to be released and then continue to execute the current task by the target node.

[0114] Specifically, the process of evaluating whether to select another computing node as the new target node can be as follows: obtain the waiting time overhead of the current task waiting for the target node to execute. Add the waiting time overhead to the predicted duration of the target node, and compare the added result with the predicted durations of other computing nodes. In this embodiment, relatively simple comparison can improve the evaluation efficiency, reduce the evaluation computation amount, and reduce the computing resources required to evaluate whether to select a new target node, which is beneficial to ensuring the scheduling and execution efficiency of the task set to be executed.

[0115] In response to the predicted duration of other computing nodes being less than the added result, use other computing nodes as the new target nodes. In response to the predicted duration of other computing nodes being not less than (i.e., higher than or equal to) the added result, use other computing nodes as the new target nodes. The specific evaluation formula can be as follows:

[0116] Equation 1-7

[0117] Equation 1-8

[0118] Wherein, represents the predicted duration for the current target node to execute the i-th target task (i.e., the current task); represents the predicted duration calculated by, for example, Equation 1-1; represents the waiting time overhead for the current task waiting for the target node to execute; represents the predicted duration for the computing node immediately following the target node in the computing node sequence of the current task to execute the current task. If the formula of Equation 1-8 holds, the current task continues to wait for the computing node ps k to execute it. Otherwise, that is, if the formula of Equation 1-8 does not hold, the next computing node that can meet the resource requirements can be found in p_select_list in ascending order of time for scheduling.

[0119] Generally speaking, as Figure 6 exemplarily shown in, Figure 6 is a schematic structural diagram of an embodiment of the computing node sequence scheduling of the present application. Figure 6 exemplarily shows the target tasks ts0 to target task tsn sorted according to the execution priority, and respectively exemplarily shows the corresponding computing node order.

[0120] S213: Let the target node execute the current task.

[0121] In this embodiment, in response to scheduling the current task to the target node, the target node can be made to execute the current task. At the same time, the target task can also be selected as the new current task according to the execution priority to achieve parallel processing of the target tasks, which can improve the processing efficiency of the task set to be executed.

[0122] It can be seen that in this embodiment, in order to find a computing node in the computing cluster where the current task can be quickly executed, after receiving the target tasks with dependencies, that is, the set of tasks to be executed, multiple target tasks can be sorted according to a certain benefit goal, and then task scheduling can be performed in the sorted order, ultimately aiming to achieve the shortest global task execution time of the set of tasks to be executed and support dynamic adjustment of the sorting goal.

[0123] Generally speaking, at the same moment in the cloud system, multiple users may submit DAG task chains simultaneously. To improve the user experience, the efficient use of system resources, and reduce resource fragmentation, etc., a task scheduling method for a single user is designed in this embodiment. By calculating the execution cost and communication cost of the target tasks in a bottom-up manner, the target tasks are sorted. The goal of this sorting process is to improve the execution compactness of the multiple target tasks during execution. Through the sorted execution priority order of the target tasks, the tasks can be scheduled to the corresponding computing nodes for execution. During scheduling, it is realized to schedule while executing, that is, after obtaining the earliest start time of the target task, adding the execution time of the task itself, and at the same time combining the execution overhead and communication overhead of the subsequent task chain, select a computing node that can make the current task execute and complete earliest for scheduling, so as to ultimately achieve the shortest execution time of multiple target tasks. And during the task scheduling process, the queuing order of the target tasks can also be dynamically adjusted according to the real-time resource usage situation of the computing nodes, through a computing node selection list, a waiting benefit function, etc., to achieve automatic optimization of the scheduling scheme.

[0124] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0125] The embodiment of the present application also provides a task scheduling device. As the name implies, the task scheduling device can schedule the computing nodes for processing target tasks. Specifically, the task scheduling device may include an input module and a scheduling module. Among them, the input module is used to input the set of tasks to be executed of a preset object. The scheduling module can execute the task scheduling method described in any of the previous embodiments. That is, the scheduling module can obtain the set of tasks to be executed of a preset object; among them, the set of tasks to be executed includes multiple target tasks with dependencies; respectively evaluate the predicted duration of each computing node in the computing cluster for executing the current task link; where the current task link includes the current task and subsequent tasks, and the subsequent tasks represent other target tasks that depend on the current task; select a computing node whose predicted duration meets the preset conditions as the target node, and schedule the current task to the target node to make the target node execute the current task.

[0126] For the description of the features in the corresponding embodiments of the task scheduling device, reference can be made to the relevant description in the corresponding embodiments of the task scheduling method, which will not be elaborated here one by one.

[0127] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above-described embodiments of the task scheduling method.

[0128] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above-described embodiments of the task scheduling method when running.

[0129] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (abbreviated as ROM), random access memory (abbreviated as RAM), mobile hard disk, magnetic disk, or optical disc and other various media that can store computer programs.

[0130] An embodiment of the present application further provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the task scheduling method.

[0131] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the task scheduling method.

[0132] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0133] The above has introduced in detail a task scheduling method, an electronic device, a computer-readable storage medium, and a computer program product provided by this application. Specific examples are used in this article to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art of this technology, without departing from the principle of this application, several improvements and modifications can still be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A task scheduling method, characterized in that, The task scheduling method includes: Obtaining a set of tasks to be executed for a preset object; wherein, the set of tasks to be executed includes multiple target tasks with dependencies; Obtaining the end task among the target tasks in the set of tasks to be executed; evaluating the link cost of the task link between the current task and the end task; selecting the task link with a higher link cost than other link costs as the current task link; Respectively evaluating the predicted duration for each computing node in the computing cluster to execute the current task link; wherein, the current task link includes the current task and successor tasks, and the successor tasks represent other target tasks that depend on the current task; Selecting the computing nodes that meet the preset conditions for the predicted duration as target nodes, scheduling the current task to the target nodes, and instructing the target nodes to execute the current task; Sorting the computing nodes based on the predicted duration to obtain a computing node sequence for each current task, wherein the predicted durations for the computing nodes in the computing node sequence to execute the current task increase sequentially; In response to the remaining computing resources of the target node not supporting the execution of the current task and the predicted duration for the computing node immediately following the target node in the computing node sequence to execute the current task being less than the superposition result, taking the computing node immediately following the target node in the computing node sequence as the new target node; the superposition result is obtained by superposing the waiting time overhead for the current task to wait for the target node to execute and its predicted duration.

2. The task scheduling method according to claim 1, wherein The link cost includes: The path length between the current task and the end task; wherein, the path length is in direct proportion to the number of target tasks between the current task and the end task; and / or, The predicted link overhead for executing the current task to the end task according to the task link.

3. The task scheduling method according to claim 2, wherein, The predicted link overhead includes the predicted backward time overhead for each computing node to execute the task link; the step of respectively evaluating the predicted duration for each computing node in the computing cluster to execute the current task link includes: Obtaining the forward time overhead from the start task to the current task; Respectively superposing the predicted backward time overheads of the computing nodes with the forward time overhead to obtain the predicted durations of the computing nodes.

4. The task scheduling method according to claim 3, wherein, The step of obtaining the forward time overhead from the start task to the current task includes: Obtaining the first time overhead for executing the start task to the forward task; wherein, the forward task represents the adjacent forward target task of the current task; Obtaining the communication time from the forward task to the current task as the second time overhead; Taking the sum value of the first time overhead and the second time overhead as the forward time overhead.

5. The task scheduling method according to claim 1, wherein Before evaluating the link cost of the task link between the current task and the end task, it further includes: Constructing a dependency topology among the target tasks; Querying the dependency topology to obtain the end task in the set of tasks to be executed; Traversing the task link from the current task to the end task in the dependency topology.

6. The task scheduling method according to claim 1, wherein Before separately evaluating the predicted duration of each computing node in the computing cluster for executing the current task link, the following steps are included: Assign a sorting factor to the target task that is positively correlated with its link cost; Compare the sorting factors of the target task in the set of tasks to be executed, sort the execution priorities of the target task, and control the execution priority of the target task to be positively correlated with the sorting factor; Select the current tasks in descending order of the execution priority in turn.

7. The task scheduling method according to claim 6, wherein The step of assigning a sorting factor to the target task that is positively correlated with its link cost includes: Independently evaluate the link cost of each computing node for executing the target task; Calculate the mean value of the link costs of each computing node as the sorting factor of the target task.

8. The task scheduling method according to claim 7, characterized in that Evaluating the link cost of the computing node for executing the target task includes: Obtain the estimated time for the computing node to execute the target task; Obtain the backward time overhead of the predicted backward task; wherein, the backward task represents the target task that depends on the current task; Obtain the pre-evaluated communication time from the target task to the backward task as the third time overhead; Add the estimated time, the backward time overhead of the backward task, and the third time overhead to obtain the backward time overhead of the target task, and use the backward time overhead of the target task as the link cost.

9. The task scheduling method according to claim 1, wherein The step of selecting the computing node with the predicted duration meeting the preset conditions as the target node includes: Compare the predicted durations of each computing node for executing the current task link; Select the computing node with the predicted duration less than the predicted durations of other computing nodes as the target node.

10. The task scheduling method according to claim 9, wherein The step of scheduling the current task to the target node further includes: Obtain the remaining computing resources of the target node; Determine whether the remaining computing resources support the execution of the current task; In response to the remaining computing resources not supporting the execution of the current task, evaluate whether to select another computing node as the new target node.

11. The task scheduling method according to claim 10, wherein The evaluation of whether to select another computing node as the new target node includes: Obtain the waiting time overhead for the current task to wait for the target node to execute; Add the waiting time overhead and the predicted duration of the target node, and compare the added result with the predicted durations of other computing nodes; In response to the predicted duration of the other computing nodes being less than the added result, use the other computing node as the new target node.

12. An electronic device, characterized in that, It includes: A memory for storing a computer program; A processor for implementing the steps of the task scheduling method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the task scheduling method according to any one of claims 1 to 11 when executed by a processor.

14. A computer program product, comprising a computer program, characterized in that, The computer program implements the steps of the task scheduling method according to any one of claims 1 to 11 when executed by a processor.

Images