Task execution information prediction method and device, medium, equipment and product
By acquiring task dependency information and resource availability, tasks can be rationally scheduled and execution completion time accurately predicted. This solves the problem of unreasonable resource allocation in big data computing scheduling, and improves task processing efficiency and the timeliness of downstream data output.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-03-27
AI Technical Summary
In big data computing and scheduling scenarios, the scheduling methods and resource allocation of data processing tasks in existing technologies are not reasonable enough, resulting in low task processing efficiency and insufficient timeliness of downstream data output.
By obtaining the task's dependency information, the target priority information of the currently executable tasks is determined, and tasks are rationally scheduled based on the remaining resources and target priority information to accurately predict the task completion time.
This improves the rationality and accuracy of task scheduling, avoids resource waste, and ensures the timely output of downstream data.
Smart Images
Figure CN121743044A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present disclosure relates to the technical field of data processing, in particular, to a task execution information prediction method and device, medium, equipment and product. BACKGROUND
[0002] In a big data computing scheduling scenario, a reasonable data processing task scheduling manner and resource allocation manner can improve the efficiency of task processing and the timeliness of downstream data output. Providing data services for the downstream is an important application in the big data scenario, in order to ensure that the required data results can be obtained in time, it is necessary to accurately predict the execution completion time of the data processing task. SUMMARY
[0003] The purpose of the present disclosure is to provide a task execution information prediction method, device, medium, equipment and product, to improve the rationality of task scheduling and accurately determine the predicted execution completion time of the task.
[0004] In order to achieve the above purpose, in a first aspect, the present disclosure provides a task execution information prediction method, the method comprising: obtaining dependency information of a plurality of tasks, the tasks being data processing tasks; determining a current executable task, the current executable task not having been executed, and a first upstream task on which the current executable task depends having been executed or the current executable task being a root node; determining target priority information of the current executable task according to a first number, a first waiting duration, and dependency information and preset priority information of the current executable task, the first number being a number of tasks currently being executed, and the first waiting duration being a duration from an actual execution completion time of the first upstream task to a current time in the case where the first upstream task exists; determining a predicted execution completion time of the current executable task according to a current remaining resource amount and the target priority information of the current executable task.
[0005] In a second aspect, the present disclosure provides a task execution information prediction device, the device comprising: an obtaining module configured to obtain dependency information of a plurality of tasks, the tasks being data processing tasks; a first determining module configured to determine a current executable task, the current executable task not having been executed, and a first upstream task on which the current executable task depends having been executed or the current executable task being a root node; The second determining module is used to determine the target priority information of the currently executable task based on the first quantity, the first waiting time, the dependency information and preset priority information of the currently executable task. The first quantity is the number of tasks currently being executed. In the case of the first upstream task, the first waiting time is the time from the actual completion time of the first upstream task to the current time. The third determining module is used to determine the predicted execution completion time of the currently executable task based on the current remaining resources and the target priority information of the currently executable task.
[0006] Thirdly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the task execution information prediction method provided in the first aspect of this disclosure.
[0007] Fourthly, this disclosure provides an electronic device, comprising: A memory on which computer programs are stored; A processor is configured to execute the computer program in the memory to implement the steps of the task execution information prediction method provided in the first aspect of this disclosure.
[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the task execution information prediction method provided in the first aspect of this disclosure.
[0009] The above technical solution determines the target priority information of the currently executable tasks based on the first quantity, the first waiting time, and the dependency and preset priority information of the currently executable tasks. When determining the target priority information, the number of currently executing tasks, the first waiting time of the currently executable tasks, and their dependency information are considered. According to the target priority information, the currently executable tasks can be scheduled more rationally. The predicted completion time of the currently executable tasks can be predicted based on the target priority information, improving the rationality of task scheduling and accurately predicting the predicted completion time of the currently executable tasks.
[0010] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description
[0011] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a task execution information prediction method according to an exemplary embodiment.
[0012] Figure 2 This is an example illustrating a dependency relationship between multiple tasks.
[0013] Figure 3 This is a block diagram illustrating a task execution information prediction device according to an exemplary embodiment.
[0014] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment.
[0015] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation
[0016] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.
[0017] Downstream data, i.e., the output of the required target data, often requires multiple data processing tasks to complete together. In related technologies, the scheduling of data processing tasks typically employs preemptive resource scheduling, meaning that data processing tasks have access to resources on a first-come, first-served basis, which may lead to unreasonable resource allocation. This application provides a task execution information prediction method, apparatus, medium, device, and product that provides more reasonable scheduling of executable tasks and accurately predicts the task completion time.
[0018] Figure 1 This is a flowchart illustrating a task execution information prediction method according to an exemplary embodiment. This method can be applied to electronic devices, which may be terminal devices or server devices. Figure 1 As shown, the task execution information prediction method includes steps 11 to 14.
[0019] Step 11: Obtain the dependency information for multiple tasks.
[0020] The task is a data processing task, used to generate corresponding data. A data processing task can also be called a data computation node. Dependency information indicates the upstream and downstream dependencies of the task; a downstream task can only use the data generated by the upstream task for further computation after the upstream task has been completed.
[0021] Figure 2 This is an example illustrating a dependency relationship between multiple tasks. For example... Figure 2As shown, multiple tasks include Task A, Task B, Task C, Task D, Task E, Task F, and Task G. Each task can also be considered a node. The direction indicated by the arrow is downstream. Task A and Task E are the root nodes, and Task G is the target task, meaning Task G is the downstream task. The downstream task is a task with an out-degree of 0, where out-degree refers to the number of edges originating from a single node. This disclosure references... Figure 2 The multiple tasks and their dependencies shown are explained and illustrated only as examples and do not constitute a limitation on the implementation.
[0022] Multiple tasks can be processed on the same device or distributed across multiple devices. If processed across multiple devices, these devices can communicate to send data. The electronic device executing the task execution information prediction method of this disclosure may be the same as or different from the device executing the data processing task.
[0023] Step 12: Determine the currently executable task.
[0024] The currently executable task has not yet been executed, and the first upstream task that the currently executable task depends on has been completed, or the currently executable task is the root node.
[0025] like Figure 2 As shown, assuming none of the tasks have started execution yet, and since the root node starts execution first, the currently executable tasks can include task A and task E. Assuming the current time is 1:00, task A actually completed execution at 1:00, and task E is currently executing. The predicted completion time of task E can be determined based on the amount of resources actually allocated to task E; the predicted completion time of task E is 1:20. Currently executable tasks include task B and task C, and the first upstream task that both task B and task C depend on is task A.
[0026] Step 13: Determine the target priority information of the current executable task based on the first quantity, the first waiting time, the dependency information of the currently executable task, and the preset priority information.
[0027] When there are multiple executable tasks, the target priority information of each currently executable task can be determined. Tasks with higher target priority information are scheduled first, and tasks with lower target priority information are scheduled later. For tasks with the same target priority, the order in which they are scheduled can be randomly set.
[0028] The first quantity refers to the number of tasks currently being executed. If a first upstream task exists, the first waiting time is the time elapsed from the actual completion time of the first upstream task to the current time. If the currently executable task depends on multiple first upstream tasks, then the actual completion time is the latest time of the actual completion time of each of the multiple first upstream tasks, i.e., the time when all the multiple first upstream tasks have completed execution. The longer the first waiting time, the sooner the currently executable task needs to be executed.
[0029] In one embodiment, the dependency information of the current executable task includes a second distance between the current executable task and the root node, a fifth number of tasks in the dependency chain, and a sixth number of downstream tasks that directly depend on the current executable task.
[0030] The second distance information refers to the number of edges between the current executable task and the root node in the directed acyclic graph corresponding to the dependency information. If there are multiple paths between the current executable task and the root node, the second distance information is the number of edges in the longest path. The shorter the second distance information, the closer the current executable task is to the root node, and the earlier it needs to be executed to ensure that downstream tasks can use the data.
[0031] A dependency link is a link from the root node to the downstream task and includes the currently executable task. If there are multiple links corresponding to the currently executable task, the dependency link can be the longest link. The more fifths in the dependency link, the more tasks are related to the currently executable task, and the earlier the currently executable task needs to be executed.
[0032] The sixth number of downstream tasks that directly depend on the currently executable task refers to the number of out-degrees of the currently executable task.
[0033] An exemplary implementation of step 13 is as follows: Determine the target priority information based on the first quantity, the first waiting time, the second distance information, the fifth quantity, the sixth quantity, the preset priority information of the currently executable task, and the preset priority information of downstream tasks that directly depend on and indirectly depend on the currently executable task.
[0034] Among them, the preset priority information of the task is information that is set in advance to reflect the importance and priority of the task. The preset priority information can be represented by a value. The higher the importance and priority of the task, the greater the corresponding preset priority information.
[0035] For example, the target priority information of the currently executable task is determined by the following formula (1): (1) Where P represents the target priority information of the currently executable task, t represents the first waiting time, w represents the first quantity, n represents the fifth quantity, x represents the second distance information, and Q... ij Let M represent the sixth quantity. M is the sum of the preset priority information of the currently executable task and the preset priority information of its downstream tasks that directly and indirectly depend on it. Taking task E as an example, tasks directly depending on E include task F, and tasks indirectly depending on E include task G. If the preset priority information of task G is M1, the preset priority information of task F is M2, and the preset priority information of task E is M3, then if the currently executable task is task F, then M is the sum of M1 and M2; if the currently executable task is task E, then M is the sum of M1, M2, and M3. In the formula, · represents a product.
[0036] in this way, This means that when determining target priority information, the number of tasks currently being executed and the waiting time of currently executable tasks are taken into account. This means that when determining target priority information, the dependency information of the currently executable tasks is taken into account. This indicates that the out-degree of the currently executable task is considered in relation to its priority. The higher the preset priority information of downstream tasks that directly or indirectly depend on the currently executable task, the more the currently executable task needs to be executed as soon as possible to provide data to the downstream tasks. Therefore, the preset priority information of both the currently executable task and its downstream tasks is considered when calculating the target priority information.
[0037] Step 14: Determine the predicted completion time of the current executable task based on the current remaining resources and the target priority information of the current executable task.
[0038] The current remaining resources can be quantified numerically. The current remaining resources can represent the remaining CPU resources, the remaining memory resources, or the total remaining resources of both CPU and memory.
[0039] Based on the current remaining resources, the target resource amount allocated to the current executable tasks can be predicted sequentially according to the target priority information of the current executable tasks. Based on the allocated target resource amount, the predicted execution completion time can be determined.
[0040] In addition, the above embodiments are based on the case where the currently executable task does not include the target task. If the currently executable task includes the target task and it is predicted that resources can be allocated to the target task at the current time, then the predicted execution completion time of the target task can be determined.
[0041] The above technical solution determines the target priority information of the currently executable tasks based on the first quantity, the first waiting time, and the dependency and preset priority information of the currently executable tasks. When determining the target priority information, the number of currently executing tasks, the first waiting time of the currently executable tasks, and their dependency information are considered. According to the target priority information, the currently executable tasks can be scheduled more rationally. The predicted completion time of the currently executable tasks can be predicted based on the target priority information, improving the rationality of task scheduling and accurately predicting the predicted completion time of the currently executable tasks.
[0042] In one embodiment, currently executable tasks include tasks that have not been executed but are currently executable, and / or tasks that have encountered execution errors and need to be re-executed.
[0043] In the above embodiments, tasks B and C are considered as tasks that are not yet executed but are currently executable. If there is another task H (not shown in the figure), which has already been executed but encountered an execution error (such as interruption or failure), the task needs to be re-executed. Therefore, currently executable tasks may also include tasks that encountered execution errors and need to be re-executed. In this disclosure, tasks that need to be re-executed typically have a longer initial waiting time. Based on the target priority information calculation method in this disclosure, a relatively high priority can be assigned to this task, thereby ensuring that re-executed tasks are scheduled with priority.
[0044] The following describes an implementation method for determining the predicted completion time of currently executable tasks. Step 14 may include: Based on the target priority information of the currently executable tasks, the current remaining resources, and the first and second resource amounts required for the execution of the currently executable tasks, determine the predicted currently executable tasks that can be allocated data processing resources and the target resource amount to be allocated to the currently executable tasks; for the predicted currently executable tasks that can be allocated data processing resources, determine the predicted execution completion time of the currently executable tasks based on the target resource amount.
[0045] Among them, the first resource quantity required for the execution of the current executable task is the minimum resource quantity required for the current executable task to be executed normally, and the second resource quantity required for the execution of the current executable task is the minimum resource quantity required when the execution time of the current executable task is the shortest execution time. The second resource quantity can be regarded as the sufficient resource quantity, that is, the task execution time is the shortest when sufficient resources are provided.
[0046] For example, an implementation for determining the predicted currently executable task that can be allocated data processing resources and the target amount of resources to be allocated to the currently executable task can be: Traverse the currently executable tasks in descending order of target priority; If the remaining resources are greater than or equal to the first resource required by the currently executable task, then the target resource amount to be allocated to the currently executable task is determined based on the remaining resources and the second resource amount. Based on the predicted target resource amount, a new remaining resource amount is determined, and the process returns to traversing the currently executable tasks in descending order of target priority, continuing until the latest remaining resource amount is less than the first resource amount required by the currently traversed executable task, at which point the traversal stops. The step of returning to traverse the currently executable tasks in descending order of target priority can be understood as continuing to traverse the next currently executable task in descending order of target priority, i.e., continuing to allocate target resources to currently executable tasks that have not yet been traversed.
[0047] For example, determining the predicted target resource amount to allocate to the currently executable task based on the remaining resource amount and the second resource amount may include: using the smaller of the remaining resource amount and the second resource amount as the target resource amount.
[0048] Continuing with the example above where the currently executable tasks include Task B and Task C, for example, if the target priority information of Task B is greater than that of Task C, the target resource amount allocated to Task B can first be predicted based on the current remaining resource amount. Assuming the current remaining resource amount is 10, the first resource amount required by Task B is 5, and the second resource amount required by Task B is 8, then the predicted target resource amount allocated to Task B is 8. Subtracting the target resource amount allocated to Task B (8) from the current remaining resource amount of 10, the new predicted remaining resource amount is 2. Next, resources are allocated to Task C. Assuming the first resource amount required by Task C is 3, the new predicted remaining resource amount cannot support the execution of Task C, so it is predicted that no resources will be allocated to Task C at this time. Assuming the first resource amount required by Task C is 1, and the second resource amount required by Task C is 5, then the predicted target resource amount allocated to Task C is 2.
[0049] In this way, based on the target priority information, the currently executable tasks can be scheduled more rationally. Based on the target priority information, the resource allocation and task completion time can be predicted, thereby improving the rationality of task scheduling and enabling accurate prediction of task execution.
[0050] In one embodiment, determining the predicted completion time of the currently executable task based on the target resource quantity includes: Based on the first execution duration of the currently executable task, the shortest execution duration of the currently executable task, the target resource amount allocated to the currently executable task, and the second resource amount required by the currently executable task, determine the predicted execution duration of the currently executable task if the target resource amount is allocated to the currently executable task, where the first execution duration is the execution duration of the currently executable task when the first resource amount is allocated to the currently executable task; based on the predicted execution duration, determine the predicted execution completion time.
[0051] The first and shortest execution times can be pre-determined based on the historical execution data of the task. For example, the predicted execution time of a currently executable task can be determined using the following formula (2): (2) Where T is the predicted execution time of the current executable task, H1 is the shortest execution time of the current executable task, H2 is the first execution time of the current executable task, R is the target resource amount allocated to the current executable task, and R2 is the second resource amount required by the current executable task.
[0052] For example, if the predicted execution time of task B is determined to be 30 minutes, then the predicted completion time of task B can be determined to be 1 hour and 30 minutes. If resources are not allocated to task C at the current time, then the predicted completion time of task C has not been determined in this iteration, but can be determined in subsequent iterations.
[0053] Thus, the execution time of a task will vary depending on the amount of resources allocated. Determining the predicted execution time based on the target amount of allocated resources can make the predicted execution time more accurate, thereby obtaining an accurate predicted completion time for the task.
[0054] The task execution information prediction method provided in this disclosure may also include: If the target task is not included in the current executable tasks, or the predicted completion time of the target task is not determined, then a task set is generated based on the first time. The first time is the predicted completion time of any task. The task set includes the predicted executable tasks. The predicted executable tasks are those that have not been executed at the first time and whose second upstream task has been completed at the first time. The target task is the downstream task among the multiple tasks. Based on the predicted target resource amount allocated to the task to be completed in the first time, determine the predicted remaining resource amount corresponding to the first time. Based on the second quantity, the second waiting time, and the dependency information and preset priority information of the predicted executable tasks, the target priority information of the predicted executable tasks is determined. The second quantity is the number of tasks predicted to be executing at the first time, and the second waiting time is the time from the predicted execution completion time or actual execution completion time of the second upstream task to the first time. Based on the predicted remaining resources and the target priority information of the predicted executable tasks, the predicted execution completion time of the predicted executable tasks is determined.
[0055] In this disclosure, the current remaining resource amount is the known remaining resource amount at the current moment. The predicted remaining resource amount is dynamically updated based on the predicted resource amount allocated to the task and the resource amount released upon task completion. Dynamic updating means that when predicting the target resource amount allocated to the task, the target resource amount is subtracted, and when predicting the resource amount released upon task completion, the released resource amount is added. For example, this predicted remaining resource amount is maintained through a variable R.
[0056] For example, the predicted completion time of task E is 1:20. Assuming that at 1:00, the predicted remaining resources are only sufficient to allocate resources to task B, and none to task C, then at the first time of this iteration, 1:20, the maintained predicted remaining resources are first added to the resources actually allocated to task E to obtain the predicted remaining resources for that first time. It is predicted that at the first time 1:20, task B is executing. The task set includes predicted executable tasks C and F. Task C depends on task A as its second upstream task, and task A's actual completion time is 1:00. Therefore, task C's second waiting time is 20 minutes. Task F depends on task E as its second upstream task, and task F's second waiting time is 0 minutes.
[0057] For each predicted executable task, the corresponding target priority information is determined. The method of predictive scheduling based on the target priority information can be referred to the above description of the current executable task.
[0058] In one embodiment, the dependency information of the predicted executable task includes a first distance between the predicted executable task and the root node, a third number of tasks in the dependency chain, and a fourth number of downstream tasks that directly depend on the predicted executable task.
[0059] The first distance information can be referred to in the above explanation of the second distance information, the third quantity can be referred to in the above explanation of the fifth quantity, and the fourth quantity refers to the number of out-degrees of the predicted executable task.
[0060] The implementation method for determining the target priority information of predictable executable tasks is as follows: The target priority information is determined based on the second quantity, the second waiting time, the first distance information, the third quantity, the fourth quantity, the preset priority information of the predicted executable task, and the preset priority information of the downstream tasks that directly depend on and indirectly depend on the predicted executable task.
[0061] For example, the target priority information for predicting executable tasks is determined by the following formula (3): (3) in, This indicates the target priority information for predicting executable tasks. Indicates the second waiting time. Indicates the second quantity. Indicates the third quantity. Indicates the first distance information. Indicates the fourth quantity. The sum of the preset priority information for predicting executable tasks and the preset priority information for downstream tasks of the directly and indirectly dependent executable tasks.
[0062] The following describes an implementation method for determining the predicted execution completion time of a predictable executable task. Specifically, the implementation method for determining the predicted execution completion time of a predictable executable task based on the predicted remaining resource amount and the target priority information of the predicted executable task can be as follows: Based on the target priority information of the predicted executable tasks, the predicted remaining resources, and the first and second resources required for the execution of the predicted executable tasks, the predicted executable tasks that can be allocated data processing resources and the target resource amount allocated to the predicted executable tasks are determined; for the predicted executable tasks that can be allocated data processing resources, the predicted execution completion time of the predicted executable tasks is determined based on the target resource amount.
[0063] Among them, the first resource amount required for the execution of the predicted executable task is the minimum resource amount required for the predicted executable task to be executed normally, and the second resource amount required for the execution of the predicted executable task is the minimum resource amount required when the execution time of the predicted executable task is the shortest execution time.
[0064] For example, an implementation for determining the predicted executable tasks that can be allocated data processing resources and the target amount of resources to be allocated to the predicted executable tasks can be: Predict executable tasks by traversing them in descending order of target priority information; If the predicted remaining resource amount is greater than or equal to the first resource amount required by the predicted executable task, then the predicted target resource amount allocated to the predicted executable task is determined based on the predicted remaining resource amount and the second resource amount. Based on the predicted target resource amount, a new predicted remaining resource amount is determined, and the process returns to the step of traversing the predicted executable tasks in descending order of target priority information, until the latest predicted remaining resource amount is less than the first resource amount required by the predicted executable task being traversed in this iteration, at which point the traversal stops. The step of returning to the step of traversing the predicted executable tasks in descending order of target priority information can be understood as continuing to traverse the next predicted executable task in descending order of target priority information, that is, continuing to allocate target resources to the predicted executable tasks that have not yet been traversed.
[0065] The process of determining the target resource amount to be allocated to the predicted executable task based on the predicted remaining resource amount and the second resource amount may include: taking the smaller value between the predicted remaining resource amount and the second resource amount as the target resource amount.
[0066] For example, the predicted executable tasks are task C and task F. The target priority information of task C is greater than that of task F. First, according to the predicted remaining resource amount at the first time, resources are allocated to task C, and then resources are allocated to task F.
[0067] In this way, based on the target priority information, the predicted executable tasks can be scheduled more rationally. The resource allocation and task completion time can be predicted based on the target priority information, thereby improving the rationality of task scheduling and enabling accurate prediction of task execution.
[0068] In one embodiment, determining the predicted execution completion time of a predicted executable task based on the target resource quantity includes: Based on the first execution duration of the predicted executable task, the shortest execution duration of the predicted executable task, the target resource amount allocated to the predicted executable task, and the second resource amount required by the predicted executable task, the predicted execution duration of the predicted executable task is determined if the target resource amount is allocated to the predicted executable task, wherein the first execution duration is the execution duration of the predicted executable task when the first resource amount is allocated to the predicted executable task; based on the predicted execution duration, the predicted execution completion time is determined.
[0069] For example, the predicted execution time of the currently executable task is determined by the following formula (4): (4) in, To predict the predicted execution time of executable tasks, To predict the shortest execution time of an executable task, To predict the first execution time of an executable task, It is the target amount of resources allocated to the currently executable tasks. It is the second amount of resources required for the currently executable task.
[0070] For example, assuming the predicted execution time of task C is 40 minutes, the predicted completion time of task C is 2:00. If the predicted execution time of task F is 60 minutes, the predicted completion time of task F is 2:20.
[0071] The task execution information prediction method provided in this disclosure may also include: If the task set generated in the previous iteration does not include the target task, or the predicted execution completion time of the target task has not been determined, then return to the step of generating the task set based on the first time, where the first time corresponding to this iteration is later than the first time corresponding to the previous iteration; If the task set generated in this iteration includes the target task, and the predicted execution completion time of the target task is determined, then the iteration stops.
[0072] The predicted completion time for task B is 1:30. At 1:30, task B is predicted to complete execution and release resources. The tasks that are predicted to be executing include task C and task F. There are no predicted executable tasks.
[0073] The predicted completion time for task C is 2:00. At the first time point of this iteration (2:00), task C is predicted to complete and release resources. Task F is predicted to be executing, and task D is predicted to be executable. With only one executable task, there is no need to calculate target priority information. Resources are allocated to task D. Assuming the predicted execution time for task D is 20 minutes, and the predicted completion time for task D is 2:20, the next iteration begins.
[0074] At 2:20, the first time corresponding to this iteration, tasks F and D are predicted to complete and release resources. The predicted executable task is task G, which is the target task. Resources are allocated to task G. Assuming that the predicted execution time of task G is 40 minutes and the predicted completion time of task G is 3:00, the predicted completion time of the target task is determined.
[0075] The above embodiment is an example where the number of target tasks is one. In practical applications, the number of target tasks can be multiple.
[0076] The above technical solution enables effective scheduling and planning of task execution based on target priority information, improving the rationality of task scheduling and avoiding resource waste and the inability to schedule important tasks in a timely manner. After determining the predicted completion time of the target task, if the predicted completion time does not meet the requirements, technical personnel can adjust the total resource volume and the preset priority information of any task to ensure the timeliness of downstream data output.
[0077] In this disclosure, the predicted execution completion time is determined in the following manner: Based on the target priority information, remaining resources, and the first and second resources required for the execution of the executable task, the predicted executable task that can be allocated data processing resources and the target resource amount allocated to the executable task are determined, wherein the first resource amount is the minimum resource amount required for the executable task to execute normally, and the second resource amount is the minimum resource amount required when the execution time of the executable task is the shortest execution time. For the executable task that is predicted to be allocated data processing resources, the predicted execution completion time of the executable task is determined based on the target resource amount.
[0078] The step of determining the predicted executable tasks that can be allocated data processing resources and the target resource amount allocated to the executable tasks based on the target priority information, the remaining resource amount, and the first and second resource amounts required for the execution of the executable tasks includes: The executable tasks are traversed in descending order of the target priority information. If the remaining resource amount is greater than or equal to the first resource amount required by the traversed executable task, then the predicted target resource amount allocated to the executable task is determined based on the remaining resource amount and the second resource amount. Based on the predicted target resource amount, a new remaining resource amount is determined, and the process returns to the step of traversing the executable tasks in descending order of the target priority information until the latest remaining resource amount is less than the first resource amount required by the executable task being traversed this time, at which point the traversal stops.
[0079] The step of determining the predicted target resource amount to be allocated to the executable task based on the remaining resource amount and the second resource amount includes: The smaller of the remaining resource quantity and the second resource quantity is taken as the target resource quantity.
[0080] Determining the predicted completion time of the executable task based on the target resource quantity includes: Based on the first execution duration, the shortest execution duration, the target resource amount, and the second resource amount, determine the predicted execution duration of the executable task if the target resource amount is allocated to the executable task, wherein the first execution duration is the execution duration of the executable task when the first resource amount is allocated to the executable task; The predicted execution completion time is determined based on the predicted execution duration.
[0081] The implementation methods for determining the predicted execution completion time have been explained above in the following ways: the implementation methods for determining the predicted execution completion time of the currently executable task and the implementation methods for determining the predicted execution completion time of the executable task.
[0082] Based on the same inventive concept, this disclosure also provides a task execution information prediction device. Figure 3 This is a block diagram illustrating a task execution information prediction device according to an exemplary embodiment, such as... Figure 3 As shown, the task execution information prediction device 30 may include: The acquisition module 31 is used to acquire dependency information of multiple tasks, wherein the tasks are data processing tasks; The first determining module 32 is used to determine the currently executable task, wherein the currently executable task has not yet been executed, and the first upstream task on which the currently executable task depends has been completed or the currently executable task is the root node; The second determining module 33 is used to determine the target priority information of the currently executable task based on the first quantity, the first waiting time, the dependency information and preset priority information of the currently executable task. The first quantity is the number of tasks currently being executed. In the case of the existence of the first upstream task, the first waiting time is the time from the actual completion time of the first upstream task to the current time. The third determining module 34 is used to determine the predicted execution completion time of the currently executable task based on the current remaining resource quantity and the target priority information of the currently executable task.
[0083] Optionally, the device 30 further includes: The set generation module is used to generate a task set based on a first time if the current executable tasks do not include the target task, or the predicted execution completion time of the target task is not determined. The first time is the predicted execution completion time of any task. The task set includes predicted executable tasks, which are tasks that are predicted not to be executed at the first time but whose dependent second upstream tasks have been completed at the first time. The target task is the most downstream task among the multiple tasks. The fourth determining module is used to determine the predicted remaining resource amount corresponding to the first time based on the predicted target resource amount allocated to the task to be completed in the first time. The fifth determining module is used to determine the target priority information of the predicted executable task based on the second quantity, the second waiting time, the dependency information of the predicted executable task, and the preset priority information, wherein the second quantity is the number of tasks predicted to be executed at the first time, and the second waiting time is the time from the predicted execution completion time or actual execution completion time of the second upstream task to the first time. The sixth determining module is used to determine the predicted execution completion time of the predicted executable task based on the predicted remaining resource amount and the target priority information of the predicted executable task.
[0084] Optionally, the device 30 further includes: The triggering module is used to re-trigger the set generation module to generate a task set based on the first time if the task set generated in the previous iteration does not include the target task, or the predicted execution completion time of the target task has not been determined. The first time corresponding to the current iteration is later than the first time corresponding to the previous iteration. If the target task is included in the task set generated in this iteration, and the predicted execution completion time of the target task is determined, the iteration stops.
[0085] Optionally, the third determining module 34 and the sixth determining module determine the predicted execution completion time through the following sub-modules: The first determining submodule is used to determine, based on the target priority information, remaining resources, and the first and second resource amounts required for the execution of the executable task, the predicted executable task that can be allocated data processing resources and the target resource amount allocated to the executable task, wherein the first resource amount is the minimum resource amount required for the executable task to execute normally, and the second resource amount is the minimum resource amount required when the execution time of the executable task is the shortest execution time; The second determining submodule is used to determine the predicted execution completion time of the executable task based on the target resource amount, for the executable task that is predicted to be allocated data processing resources.
[0086] Optionally, the first determining submodule includes: The traversal submodule is used to traverse the executable tasks in descending order of the target priority information; The third determining submodule is used to determine the predicted target resource amount to be allocated to the executable task based on the remaining resource amount and the second resource amount if the remaining resource amount is greater than or equal to the first resource amount required by the traversed executable task. The fourth determination submodule is used to determine the new remaining resource amount based on the predicted target resource amount, and trigger the traversal submodule to traverse the executable tasks in descending order of the target priority information until the latest remaining resource amount is less than the first resource amount required by the executable task traversed this time, at which point the traversal stops.
[0087] Optionally, the third determining submodule is used for: The smaller of the remaining resource quantity and the second resource quantity is taken as the target resource quantity.
[0088] Optionally, the second determining submodule is used to: Based on the first execution duration, the shortest execution duration, the target resource amount, and the second resource amount, determine the predicted execution duration of the executable task if the target resource amount is allocated to the executable task, wherein the first execution duration is the execution duration of the executable task when the first resource amount is allocated to the executable task; The predicted execution completion time is determined based on the predicted execution duration.
[0089] Optionally, the dependency information of the predicted executable task includes a first distance between the predicted executable task and the root node, a third number of tasks in the dependency chain, and a fourth number of downstream tasks that directly depend on the predicted executable task; the fifth determining module is used for: The target priority information is determined based on the second quantity, the second waiting time, the first distance information, the third quantity, the fourth quantity, the preset priority information of the predicted executable task, and the preset priority information of downstream tasks that directly depend on and indirectly depend on the predicted executable task.
[0090] Optionally, the dependency information of the currently executable task includes a second distance between the currently executable task and the root node, a fifth number of tasks in the dependency chain, and a sixth number of downstream tasks that directly depend on the currently executable task; the second determining module 33 is used to: The target priority information is determined based on the first quantity, the first waiting time, the second distance information, the fifth quantity, the sixth quantity, the preset priority information of the currently executable task, and the preset priority information of downstream tasks that directly depend on and indirectly depend on the currently executable task.
[0091] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0092] Figure 4 This is a block diagram illustrating an electronic device 700 according to an exemplary embodiment. (See diagram below.) Figure 4 As shown, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.
[0093] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in the task execution information prediction method described above. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof; therefore, the corresponding communication component 705 may include a Wi-Fi module, a Bluetooth module, or an NFC module.
[0094] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the task execution information prediction method described above.
[0095] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the task execution information prediction method described above. For example, the computer-readable storage medium may be the memory 702 including program instructions, which may be executed by the processor 701 of the electronic device 700 to complete the task execution information prediction method described above.
[0096] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, wherein the computer program, when executed by the processor, implements the steps of the task execution information prediction method described above.
[0097] Figure 5 This is a block diagram illustrating an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. (Refer to...) Figure 5 The electronic device 1900 includes a processor 1922, which may be one or more, and a memory 1932 for storing computer programs executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processor 1922 may be configured to execute the computer program to perform the task execution information prediction method described above.
[0098] Additionally, the electronic device 1900 may also include a power supply component 1926 and a communication component 1950. The power supply component 1926 can be configured to perform power management of the electronic device 1900, and the communication component 1950 can be configured to enable communication of the electronic device 1900, such as wired or wireless communication. Furthermore, the electronic device 1900 may also include an input / output (I / O) interface 1958. The electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM Linux TM etc.
[0099] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the task execution information prediction method described above. For example, the computer-readable storage medium may be the memory 1932 including program instructions, which may be executed by the processor 1922 of the electronic device 1900 to complete the task execution information prediction method described above.
[0100] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, wherein the computer program, when executed by the processor, implements the steps of the task execution information prediction method described above.
[0101] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.
[0102] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.
[0103] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.
Claims
1. A method for predicting task execution information, characterized in that, The method includes: Obtain dependency information for multiple tasks, where the tasks are data processing tasks; Determine the currently executable task, wherein the currently executable task has not yet been executed, and the first upstream task on which the currently executable task depends has been completed or the currently executable task is the root node; Based on the first quantity, the first waiting time, and the dependency information and preset priority information of the currently executable task, the target priority information of the currently executable task is determined. The first quantity is the number of tasks currently being executed. In the case of the first upstream task, the first waiting time is the time from the actual completion time of the first upstream task to the current time. Based on the current remaining resources and the target priority information of the currently executable task, the predicted execution completion time of the currently executable task is determined.
2. The method according to claim 1, characterized in that, The method further includes: If the target task is not included in the current executable tasks, or the predicted completion time of the target task is not determined, then a task set is generated based on the first time, where the first time is the predicted completion time of any task. The task set includes predicted executable tasks, where the predicted executable tasks are tasks that are predicted not to be executed at the first time but whose dependent second upstream tasks have been completed at the first time. The target task is the most downstream task among the multiple tasks. Based on the predicted target resource amount allocated to the task to be completed in the first time, determine the predicted remaining resource amount corresponding to the first time. Based on the second quantity, the second waiting time, and the dependency information and preset priority information of the predicted executable task, the target priority information of the predicted executable task is determined, wherein the second quantity is the number of tasks predicted to be executing at the first time, and the second waiting time is the time from the predicted execution completion time or actual execution completion time of the second upstream task to the first time. Based on the predicted remaining resource amount and the target priority information of the predicted executable task, the predicted execution completion time of the predicted executable task is determined.
3. The method according to claim 2, characterized in that, The method further includes: If the target task is not included in the task set generated in the previous iteration, or the predicted execution completion time of the target task has not been determined, then the step of generating the task set based on the first time is returned, wherein the first time corresponding to the current iteration is later than the first time corresponding to the previous iteration. If the target task is included in the task set generated in this iteration, and the predicted execution completion time of the target task is determined, the iteration stops.
4. The method according to claim 1 or 2, characterized in that, The predicted execution completion time is determined in the following way: Based on the target priority information, remaining resources, and the first and second resources required for the execution of the executable task, the predicted executable task that can be allocated data processing resources and the target resource amount allocated to the executable task are determined, wherein the first resource amount is the minimum resource amount required for the executable task to execute normally, and the second resource amount is the minimum resource amount required when the execution time of the executable task is the shortest execution time. For the executable task that is predicted to be allocated data processing resources, the predicted execution completion time of the executable task is determined based on the target resource amount.
5. The method according to claim 4, characterized in that, The step of determining the predicted executable tasks that can be allocated data processing resources and the target resource amount allocated to the executable tasks based on the target priority information, the remaining resource amount, and the first and second resource amounts required for the execution of the executable tasks includes: The executable tasks are traversed in descending order of the target priority information. If the remaining resource amount is greater than or equal to the first resource amount required by the traversed executable task, then the predicted target resource amount allocated to the executable task is determined based on the remaining resource amount and the second resource amount. Based on the predicted target resource amount, a new remaining resource amount is determined, and the process returns to the step of traversing the executable tasks in descending order of the target priority information until the latest remaining resource amount is less than the first resource amount required by the executable task being traversed this time, at which point the traversal stops.
6. The method according to claim 5, characterized in that, The step of determining the predicted target resource amount to be allocated to the executable task based on the remaining resource amount and the second resource amount includes: The smaller of the remaining resource quantity and the second resource quantity is taken as the target resource quantity.
7. The method according to claim 4, characterized in that, Determining the predicted completion time of the executable task based on the target resource quantity includes: Based on the first execution duration, the shortest execution duration, the target resource amount, and the second resource amount, determine the predicted execution duration of the executable task if the target resource amount is allocated to the executable task, wherein the first execution duration is the execution duration of the executable task when the first resource amount is allocated to the executable task; The predicted execution completion time is determined based on the predicted execution duration.
8. The method according to claim 2, characterized in that, The dependency information of the predicted executable task includes a first distance between the predicted executable task and the root node, a third number of tasks in the dependency link, and a fourth number of downstream tasks that directly depend on the predicted executable task. The step of determining the target priority information of the predicted executable task based on the second quantity, the second waiting time, and the dependency information and preset priority information of the predicted executable task includes: The target priority information is determined based on the second quantity, the second waiting time, the first distance information, the third quantity, the fourth quantity, the preset priority information of the predicted executable task, and the preset priority information of downstream tasks that directly depend on and indirectly depend on the predicted executable task.
9. The method according to claim 1, characterized in that, The dependency information of the currently executable task includes the second distance information between the currently executable task and the root node, the fifth number of tasks in the dependency link, and the sixth number of downstream tasks that directly depend on the currently executable task. The step of determining the target priority information of the currently executable task based on the first quantity, the first waiting time, and the dependency information and preset priority information of the currently executable task includes: The target priority information is determined based on the first quantity, the first waiting time, the second distance information, the fifth quantity, the sixth quantity, the preset priority information of the currently executable task, and the preset priority information of downstream tasks that directly depend on and indirectly depend on the currently executable task.
10. The method according to claim 1, characterized in that, The currently executable tasks include tasks that have not been executed but are currently executable, and / or tasks that have encountered execution errors and need to be re-executed.
11. A task execution information prediction device, characterized in that, The device includes: The acquisition module is used to acquire dependency information for multiple tasks, wherein the tasks are data processing tasks; The first determining module is used to determine the currently executable task, wherein the currently executable task has not yet been executed, and the first upstream task on which the currently executable task depends has been completed or the currently executable task is the root node; The second determining module is used to determine the target priority information of the currently executable task based on the first quantity, the first waiting time, the dependency information and preset priority information of the currently executable task. The first quantity is the number of tasks currently being executed. In the case of the first upstream task, the first waiting time is the time from the actual completion time of the first upstream task to the current time. The third determining module is used to determine the predicted execution completion time of the currently executable task based on the current remaining resources and the target priority information of the currently executable task.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-10.
13. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-10.
14. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program performs the steps of the method described in any one of claims 1-10.