Task scheduling method, electronic equipment, storage medium and program product

Abstract

The invention discloses a task scheduling method, electronic equipment, a storage medium and a program product, and relates to the technical field of computers, and the task scheduling method comprises the following steps: when a target task with a dependency relationship is obtained and a computing node for executing the current task is selected; the execution duration, namely the prediction duration, of the whole current task link to which the current task belongs can be considered. If yes, the computing node with the predicted duration meeting the preset condition can be selected as the target node for executing the current task, and the current task is scheduled to the target node and executed, so that the execution efficiency of the current task and subsequent tasks thereof can be comprehensively and globally improved. The technical problem of task progress lag easily caused by unreasonable task scheduling is solved, and the technical effect of reasonably planning the computational nodes for realizing the tasks to improve the task processing efficiency is achieved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task scheduling method, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Cloud computing technology is a form of computing that provides computing resources to users in the form of services through the network. Cloud computing can facilitate users to obtain computing resources and services, and can select appropriate computing nodes to implement one or more tasks sent by users. Among them, when implementing multiple tasks of users, it is necessary to allocate reasonable computing nodes to each task through a certain scheduling strategy. However, due to the unreasonable current task scheduling strategy, it is easy to have problems such as unsatisfactory task processing efficiency and even delayed task progress. Summary of the invention

[0003] The present application provides a task scheduling method, an electronic device, a computer-readable storage medium, and a computer program product, so as to at least solve the problem in the related art that unreasonable task scheduling easily leads to delayed task progress.

[0004] The present application provides a task scheduling method, which includes: obtaining a set of tasks to be executed of a preset object; wherein the set of tasks to be executed includes multiple target tasks with dependencies; respectively evaluating the predicted duration of each computing node in a computing cluster to execute a current task link; wherein the current task link includes a current task and a successor task, and the successor task represents other target tasks that depend on the current task; selecting a computing node whose predicted duration meets preset conditions as a target node, scheduling the current task to the target node, and instructing the target node to execute the current task.

[0005] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned task scheduling methods when executing the computer program.

[0006] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned task scheduling methods are implemented.

[0007] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned task scheduling methods when executed by a processor.

[0008] Through the present application, when a target task with a dependency relationship is obtained, when selecting a computing node to execute the current task, the execution duration of the entire current task link to which the current task belongs, i.e., the predicted duration, can be considered. In this way, a computing node whose predicted duration meets the preset conditions can be selected as the target node to execute the current task, and the current task can be scheduled to the target node and executed, so as to be able to comprehensively consider the overall execution efficiency of the current task and its subsequent tasks, and to reduce the scheduling process of constantly migrating each target task in the current task link between computing nodes. Therefore, the technical problem that unreasonable task scheduling easily leads to delayed task progress can be solved, and the computing nodes that implement the task can be reasonably planned to improve the technical effect of task processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 This is a structural diagram of an embodiment of a task scheduling system of the present application; Figure 2 This is a flow chart of an embodiment of a task scheduling method of the present application; Figure 3 This is a flow chart of another embodiment of the task scheduling method of the present application; Figure 4 This is a schematic diagram of the structure of an embodiment of the present application that relies on topology 1; Figure 5 This is a schematic diagram of the structure of an embodiment of a computing node sequence of the present application; Figure 6 A structural diagram of an embodiment of scheduling a computing node sequence in the present application. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0012] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0013] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0014] First, in conjunction with the specific application environment architecture or the specific hardware architecture on which the execution of the task scheduling method depends, the specific application environment architecture or the specific hardware architecture is described herein.

[0015] Specifically, see Figure 1 , Figure 1 This is a structural diagram of an embodiment of a task scheduling system of the present application.

[0016] In one embodiment, the task scheduling system can be applied to cloud computing. Cloud computing is a distributed computing that can decompose a huge data computing processing program into multiple small programs through a communication method such as a network, and then process and analyze these small programs through a computing cluster including multiple computing nodes to obtain results, and can return the results to a preset object that made the request.

[0017] Specifically, the task scheduling system may include a task scheduling device and a computing cluster. The task scheduling device may be disposed in the computing cluster or may be relatively independent of the computing cluster, which is not limited here. Figure 1 An example is given in which the task scheduling device is arranged in a computing cluster.

[0018] As the name implies, the task scheduling device is used to schedule the execution of the target task in the set of tasks to be executed fed back by the preset object, that is, it can schedule the computing nodes used to execute the target task.

[0019] The preset object is the external object of the service provided in this embodiment, which can be considered to be equivalent to the user. The task scheduling device in this embodiment and the task scheduling method described later can be to execute the task set to be executed by a single preset object for task scheduling, that is, the number of preset objects is one. It can be understood that this embodiment is the task scheduling and task execution for the task set to be executed by a user.

[0020] The set of tasks to be executed can be considered as the request sent by the user. In this embodiment, the set of tasks to be executed can include multiple target tasks with dependencies, and the target tasks can be considered as services, calculations, etc. that need to be implemented. The set of tasks to be executed can represent multiple target tasks decomposed from a request sent by a preset object; or it can represent a collection of multiple target tasks with dependencies sent by a preset object, which is not strictly limited here.

[0021] A computing node refers to a computer and its supporting equipment used in the field of computer technology, which can be used as a storage and / or computing unit in a distributed computing architecture. For example, a computing node can be a server, etc. A server is a specific IT device that provides computing power and runs software applications in a network environment. It can provide computing or application services to other clients (such as personal computers, smart phones and other terminal devices) in the network. Generally speaking, a server can have the ability to respond to service requests, provide services, and guarantee services.

[0022] A computing cluster includes multiple interconnected computing nodes. That is, multiple computing nodes form a computing cluster, and multiple computing nodes are connected to each other to achieve communication and collaboration between computing nodes. As in the example above where the computing node is a server, the computing cluster can be a server cluster, and the server cluster can be composed of multiple servers interconnected by an input / output system.

[0023] Specifically, the macro task scheduling process in this embodiment is that the computing cluster can receive a set of tasks to be executed from a preset object. The task scheduling device can schedule each target task with dependencies within the tasks to be executed, that is, schedule the execution subject and / or execution order of each target task, select a computing node that is adapted to each target task as the target node, and let the target node execute the target task.

[0024] The following is an example of the scheduling process of the task scheduling device in the present application. That is, the embodiment of the present application specifically provides a task scheduling method. The following is a detailed description of the method in conjunction with the execution flow of the task scheduling method.

[0025] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of a task scheduling method of the present application.

[0026] S101: Acquire a set of tasks to be executed of a preset object; wherein the set of tasks to be executed includes a plurality of target tasks having a dependency relationship.

[0027] In this embodiment, as described in the above examples, the target task in the set of tasks to be executed can belong to a preset object. In other words, the task scheduling method in this embodiment can be a set of tasks to be executed for a preset object, so that while achieving the target task that meets the needs of the preset object, it can deeply perceive the dependencies between the target tasks and rationalize the task scheduling process and the task execution process.

[0028] Among them, the dependency relationship can represent direct dependency or indirect dependency between two target tasks; and / or, the two target tasks can have a common directly dependent or indirectly dependent target task; and / or, the two target tasks can have a target task that directly or indirectly depends on both of them at the same time.

[0029] S102: Evaluate the predicted duration of each computing node in the computing cluster to execute the current task link respectively; wherein the current task link includes the current task and a subsequent task, and the subsequent task represents other target tasks that depend on the current task.

[0030] In this embodiment, the set of tasks to be executed of the preset object can be sent to a computing cluster, which includes multiple computing nodes. The computing nodes that execute each target task are scheduled through task scheduling, so that the computing nodes execute the corresponding computing nodes in the set of tasks to be executed.

[0031] The current task can be understood as the target task to be scheduled and executed. At the same time, as explained in the previous article, there is a certain dependency between the target tasks in the set of tasks to be executed. When intending to complete all the target tasks in the set of tasks to be executed, the current task may have a successor task, and the successor task represents other target tasks that depend on the current task. In layman's terms, the successor task can be considered as a target task that needs to be executed after the current task is completed. The successor task may require the output of the current task, and / or, the execution of the successor task after the current task is completed can improve the execution reliability, etc., which is not limited here.

[0032] That is to say, in this embodiment, when scheduling the current task, the execution time of the current task link on each computing node is considered globally. In this embodiment, it is expected that the current task and the subsequent tasks of the current task link can be implemented on the same computing node, which can improve the situation where the target tasks of the same task link are scattered on multiple computing nodes, and is conducive to improving the consistency of scheduling and execution of the current task link. In addition, in this embodiment, the predicted execution time of the current task link by the predicted computing node is used as the measurement dimension for selecting the computing node, so as to ensure the scheduling and execution efficiency of the current task and the current task link.

[0033] S103: Select a computing node whose predicted duration meets preset conditions as a target node, schedule the current task to the target node, and let the target node execute the current task.

[0034] In this embodiment, the target node represents the computing node selected to execute the current task. The computing node as the target node can be selected based on the predicted duration. After the target node is selected, the current task can be scheduled to the target node so that the target node executes the current task.

[0035] Among them, the preset condition may refer to selecting the shortest prediction duration; or, the prediction duration may be selected to be between preset duration thresholds and the remaining computing resources are sufficient, etc., which is not limited here.

[0036] It can be seen that in this embodiment, when the target task with a dependency relationship is obtained, when the computing node for executing the current task is selected, the execution duration of the entire current task link to which the current task belongs, i.e., the predicted duration, can be considered. In this way, the computing node whose predicted duration meets the preset conditions can be selected as the target node for executing the current task, and the current task can be scheduled to the target node and executed, so as to be able to comprehensively consider the overall execution efficiency of the current task and its subsequent tasks, and to reduce the scheduling process of constantly migrating each target task in the current task link from the computing nodes. Therefore, the technical problem that the unreasonable task scheduling easily leads to the delay of the task progress can be solved, and the computing nodes for realizing the task can be reasonably planned to improve the technical effect of task processing efficiency.

[0037] See also Figure 3 , Figure 3 This is a flowchart of another embodiment of the task scheduling method of the present application.

[0038] S201: Obtain a set of tasks to be executed of a preset object.

[0039] In this embodiment, the to-be-executed task set as described above may include multiple target tasks having a dependency relationship, and the multiple target tasks may belong to the same preset object.

[0040] S202: Perform resource abstraction on the computing cluster.

[0041] In this embodiment, resource abstraction processing may be performed on computing nodes in a computing cluster to facilitate subsequent rapid identification of computing nodes that can execute target tasks.

[0042] Specifically, the computing node can be abstractly expressed as processor = [P, CPU available capacity, memory available capacity, GPU core number available capacity, GPU video memory available capacity]. Among them, P can be expressed as P = {p k |p krepresents the kth processor in the system. The CPU available capacity, memory available capacity, GPU core available capacity, and GPU video memory available capacity are the capacities available for tasks on the computing node. Their values ​​can be updated in real time through the system monitoring service.

[0043] S203: Construct dependency topology between target tasks.

[0044] In this embodiment, a dependency topology such as a directed acyclic graph can be constructed between target tasks based on the dependency relationship between target tasks in the task set to be executed. In this way, relevant data can be obtained through the dependency topology, which improves the efficiency of obtaining information during task scheduling and is also conducive to improving the integrity of information acquisition, and can reduce omissions such as backward and forward tasks, omissions of task links, and misjudgment of starting and ending tasks, which is conducive to improving the reliability of task scheduling while improving the clarity of information acquisition. Figure 4 As shown in the example, Figure 4 This is a schematic diagram of the structure of an embodiment of the present application that relies on topology 1.

[0045] For example, in this embodiment, the dependency topology may be described by a four-tuple, that is, DAG=[V, E, W, C], where DAG represents the dependency topology.

[0046] V represents the set of tasks to be executed. Specifically, V can represent a set of multiple target tasks, and each target task can be represented by t1, t2, ..., t n , n is a positive integer. The task request resource information of the target task can be included in t i , i is a positive integer between 1 and n. For example, t i It can also be represented by a four-tuple, that is, T = [CPU request amount, memory request amount, GPU core number request amount, GPU video memory request amount]. CPU (Central Processing Unit) can be used as the computing and control core of the computer system, and is the final execution unit for information processing and program running. GPU (Graphics Processing Unit) is a coprocessor used to process images and graphics operations.

[0047] E is used to express the dependency relationship between target tasks, that is, E={e i,j |e i,j Indicates t i to j The communication constraint relationship}; where 1≤i<j≤n, that is, the i-th target task (t i ) and the jth target task (t j ) have a sequential execution relationship, t j In ti After the execution is completed, if t j Has multiple parent nodes (i.e., the target tasks it depends on), and can wait for all of them as t j After the parent node's target task is completed and the parent node's result data transmission arrives, t starts to execute j .

[0048] W represents the estimated execution time overhead of the target task on the computing node, that is, the estimated time. Specifically, W={w i,k |w i,k represents the estimated time of the i-th target task on the k-th computing node. Before the target task actually starts to execute, its approximate execution time on the k-th computing node can be estimated, and the estimated time can be the estimated time when the resources required by the target task can be met.

[0049] C represents the estimated information transmission time between computing nodes, that is, the communication time. Specifically, C={c i,j |c i,j represents the pre-estimated target task t i To target task t j Information transmission time}. Further, C={c i,j,k |c i,j,k represents the pre-estimated target task t on the kth computing node i To target task t j information transmission time}.

[0050] S204: Acquire the end task and the start task in the set of tasks to be executed.

[0051] In this embodiment, the starting task of the target task in the task set to be executed can be obtained, and the terminal task of the target task in the task set to be executed can be obtained. The starting task and the terminal task are also target tasks. Optionally, the dependent topology can be queried to obtain the terminal task and the starting task.

[0052] Furthermore, the forward task and the backward task of each target task may be acquired to form a forward task set and a backward task set of the target task, so as to quickly acquire relevant data in subsequent link cost evaluation and other processes.

[0053] Specifically, the forward task set of the target task can be expressed as pred(t i ). Further, pred(t i ) can be the direct predecessor target task, that is, the target task t iThe adjacent (i.e., directly dependent) forward target task of the starting task can be a set of empty tasks. i ) is used for identification, when pred(t i ) is an empty set. i It is the starting point task, equivalent to the entry task.

[0054] The backward task set of the target task can be expressed as succ(t i ). Further, succ(t i ) can be the direct successor target task, that is, the target task t i The adjacent (i.e., directly dependent) backward target task of the starting task can be a set of empty backward tasks. i ) is used for identification, when succ(t i ) is an empty set. i It is the terminal task, equivalent to the export task.

[0055] S205: Evaluate the link cost of the target task.

[0056] In this embodiment, the link cost of each computing node executing the target task can be evaluated independently, and the average of the link costs of each computing node is calculated as the ranking factor of the target task.

[0057] Optionally, the backward time cost of the target task may be used as the predicted link cost, that is, the backward time cost of the computing node executing the target task may be used as the link cost.

[0058] Specifically, the estimated time for the computing node to execute the target task can be obtained. The backward time overhead of the predicted backward task is obtained; wherein the backward task represents other target tasks that depend on the target task; the communication time from the pre-evaluated target task to the backward task is obtained as the third time overhead; the estimated time, the backward time overhead of the backward task and the third time overhead are superimposed to obtain the backward time overhead of the target task, and the backward time overhead of the target task is used as the link cost. In this way, in this embodiment, through reasonable link cost selection, the backward time overhead can be used as a link cost while also being used for the selection of subsequent target computing nodes, thereby improving the scheduling efficiency of each target task in the set of tasks to be executed as the current task. The selection of the target node will be explained in detail later.

[0059] The following is an example of a calculation formula for the backward time overhead in this embodiment: Formula 1-1 Among them, t i represents the i-th target task, i is a positive integer; p k Represents the kth computing node, where k is a positive integer; rank u (t i , p k ) represents the link cost of the kth computing node executing the i-th target task; t j represents the jth target task, and the jth target task depends on the ith target task; succ(t i ) represents the backward task set of the i-th target task, including other target tasks that depend on the i-th target task; rank u (t j , p k ) represents the link cost of the kth computing node to perform the jth target task, that is, the predicted backward task t j The backward time overhead; c i,j represents the communication time from the i-th target task to the j-th target task, i.e., the third time overhead; w i,k It indicates the estimated time for the kth computing node to execute the ith target task; max indicates the maximum value, since the target task may have multiple backward tasks.

[0060] Different from the successor tasks mentioned above, which can represent other target tasks that depend on the current task, in this embodiment, the backward task can represent the target task that depends on the evaluated target task during the evaluation of the target task. When the target task is used as the current task, the backward task that depends on it can be used as the successor task. There may be other considerations when selecting it as the successor task, which will not be described here.

[0061] S206: Assign a ranking factor that is positively correlated with the target task and its link cost.

[0062] In this embodiment, the backward time overhead of each computing node executing the target task is calculated separately as the link cost of each computing node executing the target task. Considering that the number of computing nodes that can achieve the target task may be multiple, each link cost can be processed to obtain a ranking factor. For example, the link cost can be weighted fused, fitted, averaged, etc. The following is an example of a formula for average processing of the link cost of each computing node executing the target task: Formula 1-2 Among them, hrank u (t i ) represents the ranking factor of the i-th target task; rank u (ti , p k ) represents the link cost of the k-th computing node executing the i-th target task; |P| represents the number of computing nodes in the computing cluster.

[0063] In this way, this embodiment can ensure that the target tasks can be executed in parallel in an orderly and efficient manner in the processor system, and can also make multiple target tasks to be executed have better compactness, which is conducive to making the set of tasks to be executed tend to have the characteristic of minimum execution time, while reducing the resource fragmentation of the computing cluster, thereby improving the resource utilization of the computing cluster.

[0064] Further, in order to further improve the versatility of task scheduling in this embodiment and improve the efficiency of task scheduling. The processor type may include isomorphic processors and heterogeneous processors. That is to say, the processor type of the computing cluster can be obtained, and when the processor type of the computing cluster is a heterogeneous processor, it can be evaluated according to the ranking factor method such as the example of formula 1-1 and formula 1-2. When the processor type of the computing cluster is a homogeneous processor, the average execution time of the computing nodes of the computing cluster on each processor can be evaluated, and the maximum value of the backward time overhead of the backward task that depends on the target task and the average communication time of the target task and the backward task can be evaluated. The sum of the maximum value and the average execution time is used as the link cost of the target task, so that the target task can be executed in parallel in an orderly and efficient manner in the processor system, and multiple target tasks to be executed can also have better compactness, which is conducive to making the set of tasks to be executed have the characteristic of tending to the minimum execution time, while reducing the resource fragmentation of the computing cluster, thereby improving the resource utilization of the computing cluster. At the same time, by calculating the average execution time of the node, it is also possible to reduce the execution time of each computing node separately, which can significantly improve the scheduling efficiency.

[0065] The evaluation can be performed according to the following ranking factor calculation method.

[0066] Formula 2-1 Among them, rank u (t i ) represents the link cost of the i-th target task under the homogeneous processor type; represents the average execution time of each computing node processing the i-th target task; rank u (t j ) represents the link cost of the jth target task under the homogeneous processor type; represents the average communication time from the i-th target task to the j-th target task. u (t i ) as the sorting factor.

[0067] The following as well as The calculation formula is given as an example: Formula 2-2 Formula 2-3 Where P represents the computing cluster, p represents the computing node; ct i,p represents the execution time of a computing node to execute the i-th target task; |p| represents the number of computing nodes with cumulative execution time in the denominator; represents the communication time from the computing node executing the i-th target task to the computing node executing the j-th target task, that is, p i represents the computing node that executes the i-th target task, p j represents the computing node executing the jth target task; |P|*|P| represents the number of communication paths between the computing node executing the i-th target task and the computing node executing the j-th target task in the numerator of Formula 2-3.

[0068] S207: Sort the execution priorities of the target tasks based on the sorting factors.

[0069] In this embodiment, the execution priority of the target task can be sorted by comparing the sorting factors of the target tasks in the task set to be executed, and the execution priority of the target task is controlled to be positively correlated with the sorting factor. In layman's terms, in this embodiment, the target task with a higher execution priority can be scheduled and executed earlier than the target task with a lower priority. In this implementation, it is expected to give priority to scheduling and executing the target tasks and their task links that consume more link costs, so that they can be scheduled and executed first, and the time for their execution to be completed is moved forward, so as to be able to advance the execution completion time of the task set to be executed.

[0070] Furthermore, after the execution priorities of the target tasks are sorted, the execution order of the target tasks after the execution priorities are sorted can also be reviewed. As the name implies, target tasks with higher execution priorities can be scheduled for execution earlier, so before scheduling and executing the current task, the current task and other target tasks that have not yet been scheduled and executed can be located in the dependency topology, and based on the located position, it can be confirmed whether the current task depends on other target tasks that have not yet been scheduled, so as to realize a review of whether the execution priority of the target task is reliable, which is conducive to reducing the occurrence of layer jumping and the like, and is conducive to ensuring the reliability of scheduling and execution of the task set to be executed. When it is confirmed that the current task does not depend on other unscheduled target tasks, the current task can continue to be scheduled and executed. When it is confirmed that the current task depends on other unscheduled target tasks, the execution priority of the current task is reduced to a level lower than the execution priority of the target task on which it depends, and the current task is reacquired based on the execution priority.

[0071] S208: Acquire the current task based on the execution priority.

[0072] In this embodiment, the current tasks may be selected in descending order of execution priority, so as to advance the execution completion time of the to-be-executed task set.

[0073] S209: Selecting a current task link of the current task based on the link cost.

[0074] In this embodiment, specifically, the task link from the current task to the terminal task in the dependency topology can be traversed. And the link cost of the task link between the current task and the terminal task is evaluated respectively. The task link with a link cost higher than other link costs is selected as the current task link. In layman's terms, in this embodiment, the task link with the highest link cost can be used as the current task link, so as to give priority to the allocation of computing nodes to the current task link with the highest link cost, which is beneficial to improve the scheduling and execution start time and efficiency of the current task link. After completing the scheduling of the current task, that is, the current task link, other task links with relatively low link costs can also be scheduled, and then the current task link with the highest link cost can be scheduled in advance, so as to help globally advance the execution completion time of the task set to be executed.

[0075] Optionally, the link cost may include: the path length between the current task and the end task; wherein the path length is in direct proportion to the number of target tasks between the current task and the end task. And / or, the predicted link cost from the current task to the end task is executed according to the task link.

[0076] For example, the link cost can be represented by the path length alone. Usually, the path length is positively correlated with the link cost of the task link. Representing the link cost by the path length is conducive to reducing the efficiency of link cost calculation. Representing the link cost by predicting the link overhead is conducive to improving the accuracy of the link cost, which can improve the rationality of the planned task scheduling strategy, thereby helping to improve the execution efficiency of the task set to be executed.

[0077] Alternatively, the path length and the predicted link cost may be considered to comprehensively evaluate the link cost. The path length and the predicted link cost may be superimposed, weighted, integrated, or fitted, etc., which are not limited here.

[0078] Optionally, the predicted link overhead includes the predicted backward time overhead of each computing node executing the task link.

[0079] As the link cost calculated in formula 1-1 above, rank u (t i , p k )The backward time overhead of the i-th target node calculated is used as the link cost.

[0080] S210: Evaluate the predicted duration of each computing node in the computing cluster to execute the current task link respectively.

[0081] In this embodiment, the current task link may include a current task and a subsequent task, where the subsequent task represents other target tasks that depend on the current task.

[0082] Specifically, the forward time overhead from the execution of the starting task to the current task can be obtained; the predicted backward time overhead of each computing node is superimposed with the forward time overhead to obtain the predicted duration of each computing node, so as to globally consider the time overhead of the computing node executing the current task link.

[0083] Furthermore, the first time cost from executing the starting task to the forward task can be obtained; wherein the forward task represents the adjacent forward target task of the current task; the communication time from the forward task to the current task is obtained as the second time cost; and the sum of the first time cost and the second time cost is taken as the forward time cost. In this way, the actual forward task execution time can be combined to participate in the scheduling process of the current task, which can improve the reliability and accuracy of the current task scheduling process. Optionally, the first time cost of actually executing the forward task can be compared with the estimated time cost, so as to use the difference between the two to update the estimated time of the estimated execution target task, so as to improve the accuracy of the estimated time in the process of task scheduling and execution, so as to improve the accuracy of the link cost and the predicted duration, and thus improve the reliability of the task scheduling in this embodiment.

[0084] Formula 1-3 Among them, EST(t i , p k ) represents the forward time cost of executing to the current task; t h represents the hth target task, the i-th target task depends on the h-th target task, that is, the h-th target task is the forward task of the i-th target task; pred(t i ) represents the forward task set of the i-th target task; AFT(t h ) represents the first time cost from the execution of the starting task to the completion of the hth target task; c h,i It represents the communication time from the hth target task to the ith target task, that is, the second time overhead.

[0085] Among them, EST(t i , p k ) can also be considered as the earliest start time of the i-th target task, that is, when calculating the link cost from the starting task to the k-th processor, the maximum path or cost in the preceding node can be taken, so that when multiple target tasks are executed in parallel, the earliest start time of executing the target task is determined by the target task with the longest execution time in the same batch, which can improve the reliability of the estimated earliest start time of the target task.

[0086] S211: Select a computing node whose predicted duration meets preset conditions as a target node.

[0087] In this embodiment, the predicted duration of each computing node to execute the current task link is compared. The computing node with a predicted duration less than the predicted duration of other computing nodes is selected as the target node. This is equivalent to scheduling the current task to the computing node considered optimal in this embodiment, that is, the time when the current task link to which the current task belongs is executed and completed at the target node can be earlier than the execution completion time at other computing nodes, thereby reasonably allocating computing nodes to the current task to improve the scheduling and execution efficiency of the current task link. The following is an example of the formula for calculating the predicted duration in this embodiment: Formula 1-4 Among them, EFT(t i , p k ) represents the time when the i-th target task is completed; w i,k It represents the estimated time for the k-th computing node to execute the i-th target task. k is a positive integer and can be calculated for each computing node in the computing cluster.

[0088] Formula 1-5 Among them, LDFE(t i , pk ) represents the backward time overhead of the k-th computing node executing the backward task of the i-th target task.

[0089] Formula 1-6 Among them, p ps0 Indicates the predicted duration of the computing node selected as ps0, where ps0 indicates the computing node used as the target node.

[0090] like Figure 5 As shown in the example, Figure 5 A schematic diagram of the structure of an embodiment of a computing node sequence of the present application. Figure 5 In the example, the p_select_list (i.e., the computing node sequence) is obtained by sorting the computing nodes based on the predicted duration. Among them, the predicted durations of the computing nodes corresponding to ps0 to ps6 for executing the current task / target task increase in sequence.

[0091] S212: Schedule the current task to the target node.

[0092] In this embodiment, the current task may be scheduled to a target node in the expectation that the target node will execute the current task.

[0093] Furthermore, the remaining computing resources of the target node are obtained to determine whether the remaining computing resources support the execution of the current task, thereby ensuring the stability of the computing cluster and taking into account the reliable execution of the current task.

[0094] In response to whether the remaining computing resources support the execution of the current task, the target node is instructed to execute the current task.

[0095] In response to the remaining computing resources not supporting the execution of the current task, it is evaluated whether to select another computing node as a new target node. When it is determined that another computing node is selected as the new target node, the current task is scheduled to the new target node for execution. When it is determined that no computing node is selected as the new target node, the computing resources of the target node are released, and the current task continues to be executed by the target node.

[0096] Specifically, the process of evaluating whether to select another computing node as a new target node may be to obtain the waiting time overhead of the current task waiting for the target node to execute. The waiting time overhead is superimposed with the predicted duration of the target node, and the superimposed result is compared with the predicted duration of other computing nodes. In this embodiment, the evaluation efficiency can be improved by relatively simple comparison, and the evaluation calculation amount can also be reduced, which can reduce the computing resources required for evaluating whether to select a new target node, so as to facilitate the scheduling and execution efficiency of the task set to be executed.

[0097] In response to the predicted duration of other computing nodes being less than the superposition result, the other computing nodes are used as new target nodes. In response to the predicted duration of other computing nodes being not less than (i.e., higher than or equal to) the superposition result, the other computing nodes are used as new target nodes. The specific evaluation formula can be as follows: Formula 1-7 Formula 1-8 in, Indicates the predicted duration of the current target node to execute the i-th target task (i.e., the current task); represents the predicted duration calculated by formula 1-1; Indicates the waiting time overhead of the current task waiting for the target node to execute; Indicates the predicted duration of the current task to be executed by the computing node located one position after the target node in the computing node sequence of the current task. If the formula 1-8 is established, the current task continues to wait for the computing node ps k Otherwise, the formula 1-8 does not hold, and the next computing node that can meet the resource requirements can be found in the p_select_list in the order of increasing time for scheduling.

[0098] In layman's terms, Figure 6 As shown in the example, Figure 6 A structural diagram of an embodiment of scheduling a computing node sequence in the present application. Figure 6 In the figure, target tasks ts0 to tsn sorted according to execution priority are given as examples, and the corresponding computing node sequences are given as examples respectively.

[0099] S213: Instruct the target node to execute the current task.

[0100] In this embodiment, in response to dispatching the current task to the target node, the target node can be made to execute the current task. At the same time, the target task as the new current task can also be selected according to the execution priority to achieve parallel processing of the target tasks, which can improve the processing efficiency of the task set to be executed.

[0101] It can be seen that in this embodiment, in order to find a computing node in the computing cluster where the current task can be quickly executed, after receiving a target task with a dependency relationship, that is, a set of tasks to be executed, multiple target tasks can be sorted according to a certain benefit target, and then the tasks can be scheduled in the sorted order, ultimately tending to achieve the goal of minimizing the global task execution time of the set of tasks to be executed and supporting dynamic adjustment of the sorting.

[0102] Generally speaking, in the cloud system, multiple users may submit DAG task chains at the same time at the same time. In order to improve the user experience and the efficient use of system resources, reduce resource fragmentation, etc., a task scheduling method for a single user is designed in this embodiment. The execution cost and communication cost of the target task are calculated in a bottom-up manner, and the target task is sorted. The goal of the sorting process is to improve the execution compactness of the target task when multiple target tasks are executed. Through the arranged target task execution priority order, the task can be scheduled to the corresponding computing node for execution. During scheduling, the scheduling is implemented while executing, that is, after obtaining the earliest start time of the target task, the execution time of the task itself is added, and the execution overhead and communication overhead of the subsequent task chain are combined. The computing node that can make the current task complete the earliest is selected for scheduling, so that the execution time of multiple target tasks can be achieved in the end. The shortest execution time. In addition, in the process of task scheduling, the queue order of the target task can also be dynamically adjusted according to the real-time resource usage of the computing node through the computing node selection list and the waiting benefit function, so as to realize the automatic tuning of the scheduling scheme.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0104] The embodiment of the present application also provides a task scheduling device. As the name implies, the task scheduling device can schedule the computing nodes that process the target tasks. Specifically, the task scheduling device may include an input module and a scheduling module. Among them, the input module is used to input the set of tasks to be executed of the preset object. The scheduling module can execute the task scheduling method as described in any of the above embodiments. That is, the scheduling module can obtain the set of tasks to be executed of the preset object; wherein the set of tasks to be executed includes multiple target tasks with dependencies; respectively evaluate the predicted duration of each computing node in the computing cluster to execute the current task link; wherein the current task link includes the current task and the subsequent task, and the subsequent task represents other target tasks that depend on the current task; select the computing node whose predicted duration meets the preset conditions as the target node, schedule the current task to the target node, and let the target node execute the current task.

[0105] The description of the features in the embodiment corresponding to the task scheduling device can refer to the relevant description of the embodiment corresponding to the task scheduling method, which will not be repeated here.

[0106] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned task scheduling method embodiments.

[0107] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned task scheduling method embodiments when running.

[0108] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0109] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned task scheduling method embodiments are implemented.

[0110] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned task scheduling method embodiments are implemented.

[0111] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0112] The above is a detailed introduction to a task scheduling method, electronic device, computer-readable storage medium and computer program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A task scheduling method, characterized in that: The task scheduling method comprises: Acquire a set of tasks to be executed of a preset object; wherein the set of tasks to be executed includes a plurality of target tasks having a dependency relationship; Evaluate the predicted duration of each computing node in the computing cluster to execute the current task link respectively; wherein the current task link includes the current task and the subsequent task, and the subsequent task represents other target tasks that depend on the current task; The computing node whose predicted duration meets the preset conditions is selected as the target node, the current task is scheduled to the target node, and the target node is made to execute the current task.

2. The task scheduling method according to claim 1, characterized in that: The step of respectively evaluating the predicted duration of each computing node in the computing cluster to execute the current task link includes: Obtaining the terminal task in the target task in the set of tasks to be executed; Evaluate the link cost of the task link between the current task and the endpoint task; A task link whose link cost is higher than other link costs is selected as the current task link.

3. The task scheduling method according to claim 2, characterized in that: The link cost includes: The path length between the current task and the terminal task; wherein the path length is in direct proportion to the number of target tasks between the current task and the terminal task; and / or, The predicted link cost from the current task to the destination task is executed according to the task link.

4. The task scheduling method according to claim 3, characterized in that: The predicted link overhead includes the predicted backward time overhead of each computing node executing the task link; and the respectively evaluating the predicted duration of each computing node in the computing cluster executing the current task link includes: Obtain the forward time cost from the execution of the starting task to the current task; The predicted backward time overhead of each computing node is respectively superimposed on the forward time overhead to obtain the predicted duration of each computing node.

5. The task scheduling method according to claim 4, characterized in that: The forward time cost from the start task execution to the current task is obtained by: Obtaining the first time cost from executing and completing the starting task to the forward task; wherein the forward task represents the adjacent forward target task of the current task; Acquire the communication time from the forward task to the current task as the second time cost; The sum of the first time overhead and the second time overhead is used as the forward time overhead.

6. The task scheduling method according to claim 2, characterized in that: Before evaluating the link cost of the task link between the current task and the end task, the step further includes: Constructing a dependency topology between the target tasks; Querying the dependency topology to obtain the terminal task in the set of tasks to be executed; Traverse the task link from the current task to the end task in the dependency topology.

7. The task scheduling method according to claim 1, characterized in that: The step of respectively evaluating the predicted duration of each computing node in the computing cluster to execute the current task link includes: Assigning a ranking factor that is positively correlated to the target task and its link cost; Comparing the ranking factors of the target tasks in the set of tasks to be executed, ranking the execution priorities of the target tasks, and controlling the execution priorities of the target tasks to be positively correlated with the ranking factors; The current tasks are selected in descending order of the execution priorities.

8. The task scheduling method according to claim 7, characterized in that: The ranking factors that give the target task a positive correlation with its link cost include: Independently evaluating the link cost of each computing node in executing the target task; The average of the link costs of the computing nodes is calculated as a ranking factor of the target task.

9. The task scheduling method according to claim 8, characterized in that: Evaluating the link cost of the computing node executing the target task includes: Obtaining an estimated time for the computing node to execute the target task; Obtaining a predicted backward time overhead of a backward task; wherein the backward task represents a target task that depends on the current task; Acquire the pre-estimated communication time from the target task to the backward task as a third time cost; The estimated time, the backward time overhead of the backward task and the third time overhead are superimposed to obtain the backward time overhead of the target task, and the backward time overhead of the target task is used as the link cost.

10. The task scheduling method according to claim 1, characterized in that: The selecting the computing node whose predicted duration meets a preset condition as the target node comprises: Comparing the predicted duration of each computing node executing the current task link; A computing node whose predicted duration is shorter than the predicted durations of other computing nodes is selected as the target node.

11. The task scheduling method according to claim 10, characterized in that: The dispatching the current task to the target node further comprises: Obtaining remaining computing resources of the target node; Determining whether the remaining computing resources support the execution of the current task; In response to the remaining computing resources not supporting the execution of the current task, evaluating whether to select another computing node as a new target node.

12. The task scheduling method according to claim 11, characterized in that: The evaluation of whether to select another computing node as a new target node includes: Obtaining the waiting time overhead of the current task waiting for the target node to execute; Superimposing the waiting time overhead and the predicted duration of the target node, and comparing the superimposed result with the predicted durations of other computing nodes; In response to the predicted duration of the other computing node being less than the superposition result, the other computing node is used as the new target node.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the task scheduling method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the task scheduling method according to any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 12 are implemented.

Images