Task scheduling method and device, computer device and readable storage medium

Abstract

This application relates to a task scheduling method, apparatus, computer device, and readable storage medium. The method includes: acquiring first task information of a target task, and task execution information, status information, and historical information of multiple executors; the historical information includes second task information of multiple historical tasks; for each executor, filtering candidate tasks from the multiple historical tasks based on the similarity between the first and second task information; predicting the estimated completion time of the executor's task based on the second task information of the multiple candidate tasks and the executor's task execution information; obtaining a task scheduling score for the executor based on its estimated completion time and status information; and determining a scheduling executor from the multiple executors based on their respective task scheduling scores, wherein the scheduling executor is used to process the target task. This method can improve the accuracy of task scheduling.

Description

Technical Field

[0001] This application relates to the field of distributed task scheduling technology, and in particular to a task scheduling method, apparatus, computer device, and readable storage medium. Background Technology

[0002] In a task scheduling system, the scheduling component, as the core coordination component, is responsible for dynamically allocating massive computing tasks to multiple executors for processing. The scheduling component monitors the cluster resource status, task queue load, and the health of each executor in real time, and performs intelligent distribution based on preset strategies (such as round-robin or random).

[0003] However, when making task scheduling decisions based on existing strategies, the only basis for decision-making is the static state information of whether the executor is currently idle, which makes the scheduling accuracy of tasks low. Summary of the Invention

[0004] Therefore, it is necessary to provide a task scheduling method, apparatus, computer device, and readable storage medium that can improve the accuracy of task scheduling in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a task scheduling method, comprising: acquiring first task information of a target task, and task execution information, status information, and historical information of multiple executors; the historical information includes second task information of multiple historical tasks; for each executor, filtering candidate tasks from the multiple historical tasks based on the similarity between the first task information and the second task information; predicting the expected completion time of the executor's task based on the second task information of the multiple candidate tasks and the task execution information of the executor; obtaining a task scheduling score for the executor based on the expected completion time of the executor's task and the status information; and determining a scheduling executor from the multiple executors based on the task scheduling scores of the multiple executors, wherein the scheduling executor is used to process the target task.

[0006] In one embodiment, the second task information includes the historical execution time of the task, and the task execution information includes the execution time of the currently executing task of the executor; the step of predicting the expected completion time of the executor's task based on the second task information of each of the multiple candidate tasks and the task execution information of the executor includes: determining the historical average execution time among the historical execution times of the multiple candidate tasks, and a first time difference between the historical average execution time and the executed time; calling a preset Bayesian execution time prediction model to predict the predicted execution time of the currently executing task; determining a second time difference between the predicted execution time and the executed time; determining the maximum value between the first time difference and the second time difference as the remaining execution time of the executor's task; for each waiting task in the executor's processing queue, determining the sum of the expected execution times corresponding to each waiting task as the waiting execution time of the executor; and determining the sum of the remaining execution time of the task and the waiting execution time as the expected completion time of the executor's task.

[0007] In one embodiment, obtaining the task scheduling score of the executor based on the expected task completion time of the executor and the status information includes: determining the average completion time among the expected task completion times of the plurality of executors, and the average queue length among the queue lengths of the respective processing queues of the plurality of executors; obtaining the expected completion time score of the executor based on the average completion time and the expected task completion time of the executor; obtaining the load balancing score of the executor based on the average queue length and the queue length of the processing queue of the executor; and obtaining the task scheduling score of the executor based on the expected completion time score and the load balancing score.

[0008] In one embodiment, the status information includes resource utilization rate, and the historical information includes the task execution results of each of the plurality of historical tasks; obtaining the task scheduling score of the executor based on the expected completion time score and the load balancing score includes: determining the resource utilization rate score of the executor based on the resource utilization rate; determining the task success rate corresponding to a target historical task among the plurality of historical tasks based on the task execution results of each of the plurality of historical tasks; the target historical task and the target task have the same task type; determining the historical reliability score of the executor as the ratio of the task success rate to the average task success rate corresponding to the plurality of executors; and performing a weighted summation of the expected completion time score, the load balancing score, the resource utilization rate score, and the historical reliability score to obtain the task scheduling score of the executor.

[0009] In one embodiment, obtaining the expected completion time score of the executor based on the average completion time and the expected completion time of the executor's task includes: determining a first ratio between the negative of the expected completion time of the task and the average completion time; and processing the first ratio using an exponential function to obtain the expected completion time score of the executor.

[0010] In one embodiment, the status information includes the queue length of the processing queue; the method further includes: determining, from the plurality of executors, executors whose queue length of the processing queue is less than or equal to a preset length threshold as target executors; the number of target executors is at least two; the step of filtering candidate tasks from the plurality of historical tasks for each executor based on the similarity between the first task information and the second task information includes: for each target executor, filtering candidate tasks from the plurality of historical tasks based on the similarity between the first task information and the second task information.

[0011] In one embodiment, the method further includes: when the queue length of the processing queues of the plurality of executors is greater than the preset length threshold, determining the executor with the shortest processing queue length from the plurality of executors as the scheduling executor.

[0012] Secondly, this application provides a task scheduling apparatus, the apparatus comprising: an acquisition module, configured to acquire first task information of a target task, and task execution information, status information, and historical information of a plurality of executors; the historical information including second task information of a plurality of historical tasks; a filtering module, configured to, for each executor, filter candidate tasks from the plurality of historical tasks based on the similarity between the first task information and the second task information; a processing module, configured to, based on the second task information of the plurality of candidate tasks and the task execution information of the executor, predict the expected completion time of the executor's task; an analysis module, configured to, based on the expected completion time of the executor's task and the status information, obtain a task scheduling score for the executor; and a determination module, configured to, based on the task scheduling scores of the plurality of executors, determine a scheduling executor from the plurality of executors, the scheduling executor being used to process the target task.

[0013] Thirdly, this application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring first task information of a target task, and task execution information, status information, and historical information of multiple executors; the historical information includes second task information of multiple historical tasks; for each executor, filtering candidate tasks from the multiple historical tasks based on the similarity between the first task information and the second task information; predicting the expected completion time of the executor's task based on the second task information of the multiple candidate tasks and the task execution information of the executor; obtaining the task scheduling score of the executor based on the expected completion time of the executor's task and the status information; and determining a scheduling executor from the multiple executors based on the task scheduling scores of the multiple executors, the scheduling executor being used to process the target task.

[0014] Fourthly, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the following steps: acquiring first task information of a target task, and task execution information, status information, and historical information of a plurality of executors; the historical information includes second task information of a plurality of historical tasks; for each executor, filtering candidate tasks from the plurality of historical tasks based on the similarity between the first task information and the second task information; predicting the expected completion time of the executor's task based on the second task information of the plurality of candidate tasks and the task execution information of the executor; obtaining the task scheduling score of the executor based on the expected completion time of the executor's task and the status information; and determining a scheduling executor from the plurality of executors based on the task scheduling scores of the plurality of executors, wherein the scheduling executor is used to process the target task.

[0015] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps: acquiring first task information of a target task, and task execution information, status information, and historical information of a plurality of executors; the historical information includes second task information of a plurality of historical tasks; for each executor, filtering candidate tasks from the plurality of historical tasks based on the similarity between the first task information and the second task information; predicting the expected completion time of the executor's task based on the second task information of the plurality of candidate tasks and the task execution information of the executor; obtaining a task scheduling score for the executor based on the expected completion time of the executor's task and the status information; and determining a scheduling executor from the plurality of executors based on the task scheduling scores of the plurality of executors, the scheduling executor being used to process the target task.

[0016] The aforementioned task scheduling method, apparatus, computer device, and readable storage medium acquire first task information of the target task, as well as task execution information, status information, and historical information of multiple executors. The historical information includes second task information for each of the multiple historical tasks. For each executor, candidate tasks are selected from multiple historical tasks based on the similarity between the first and second task information. Thus, by introducing task similarity as a selection dimension, the task scheduling process can identify the executor's past ability and efficiency in handling similar tasks, providing historical experience for scheduling decisions. By predicting the executor's expected task completion time based on the second task information of multiple candidate tasks and the executor's task execution information, historical execution capabilities are correlated with real-time execution capabilities, achieving accurate prediction of future performance. By obtaining the executor's task scheduling score based on its expected completion time and status information, and then determining the scheduling executor from among multiple executors based on their respective task scheduling scores, the scheduling executor is used to process the target task. Therefore, by ensuring that the target task is assigned to the most suitable and efficient executor, the accuracy and efficiency of task scheduling can be improved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a diagram illustrating the application environment of a task scheduling method in one embodiment;

[0019] Figure 2 This is a flowchart illustrating a task scheduling method in one embodiment;

[0020] Figure 3 This is a flowchart illustrating the process of predicting the estimated completion time of an executor's task based on the second task information of each of multiple candidate tasks and the task execution information of the executor in one embodiment.

[0021] Figure 4 This is a flowchart illustrating the process of obtaining the task scheduling score of an executor based on the expected completion time and status information of the executor's task in one embodiment.

[0022] Figure 5 This is a flowchart illustrating the process of obtaining the task scheduling score of the executor based on the expected completion time score and the load balancing score in one embodiment.

[0023] Figure 6 This is an application scenario of the task scheduling method in another embodiment;

[0024] Figure 7 This is a flowchart illustrating the task scheduling method in another embodiment;

[0025] Figure 8 This is a structural block diagram of a task scheduling device in one embodiment;

[0026] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] It should be noted that the terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" as used in this application refers to two or more. The term "and / or" as used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0029] Existing task scheduling frameworks (such as XXL-JOB) provide various routing strategies, including First-in-First-Out (FIRST) executor selection, Round-Robin (ROUND) scheduling, Random (RANDOM) selection, Consistent Hashing Routing (CONSISTENT_HASH), Least Frequently Used (LHF) strategy, Least Recently Used (LRU) strategy, Busy Overload Transfer (BUSYOVER) strategy, Failover strategy, and Sharding Broadcast strategy. However, existing strategies have the following problems:

[0030] 1. Insufficient static judgment capability. For example, the existing strategy can only determine whether the executor is currently idle, but cannot predict when the executor will complete the current task, which makes it impossible to make the optimal choice when all executors are busy.

[0031] 2. Lack of historical learning ability, such as the inability to use historical execution time data to optimize scheduling decisions, resulting in low scheduling efficiency.

[0032] 3. The load assessment is too simplistic, such as only considering whether the executor is busy, without comprehensively considering multiple factors such as queue length, task execution time, and resource utilization.

[0033] 4. Unable to predict future states, such as the load state of the actuator at future moments, making it impossible to make optimization decisions in advance.

[0034] 5. Insufficient optimization of waiting time; for example, when all executors are busy, it is impossible to select the executor with the shortest expected waiting time.

[0035] As can be seen from the above, the technical problems that need to be solved based on the existing strategy are: the lack of execution time prediction capability in the existing task scheduling system, which leads to the inability to accurately select the execution with the shortest expected completion time when all executors are busy, resulting in excessively long average task waiting time, low system throughput, unbalanced executor load, and low resource utilization.

[0036] In view of this, executor status data can be collected in real time. By calling the preset Bayesian execution time prediction model, and taking into account multiple dimensions such as task feature similarity, executor queue status, and resource utilization, the executor with the shortest expected completion time can be intelligently selected for task scheduling, thereby achieving adaptive intelligent scheduling.

[0037] In view of this, this application provides a task scheduling method, which can be applied to tasks such as... Figure 1 The application environment shown is illustrated. In this environment, the task scheduling system 10 is communicatively connected to the distributed executor cluster 20. The distributed executor cluster 20 includes multiple executors, and the task scheduling system 10 includes a scheduling component 102.

[0038] Specifically, the scheduling component 102 is used to obtain the first task information of the target task, as well as the task execution information, status information, and historical information of each of the multiple executors; the historical information includes the second task information of each of the multiple historical tasks; for each executor, candidate tasks are selected from the multiple historical tasks based on the similarity between the first task information and the second task information, and the estimated task completion time of the executor is predicted based on the second task information of each of the multiple candidate tasks and the task execution information of the executor; the task scheduling score of the executor is obtained based on the estimated task completion time and status information of the executor, and then the scheduling executor is determined from the multiple executors based on the task scheduling scores of each of the multiple executors, and the scheduling executor is used to process the target task.

[0039] In one exemplary embodiment, such as Figure 2 As shown, a task scheduling method is provided, which can be applied to... Figure 1 Taking the scheduling component 102 as an example, the following steps are included:

[0040] S202, obtain the first task information of the target task, as well as the task execution information, status information and historical information of each of the multiple executors; the historical information includes the second task information of each of the multiple historical tasks.

[0041] The target task refers to the task that will be processed by the executor, and the first task information refers to the task information related to the target task. The first task information includes, but is not limited to: task identifier, task type, task parameters, task data volume, and task priority.

[0042] Task execution information refers to information related to the executor's currently executing task, which is the task the executor is currently processing. Task execution information includes, but is not limited to: the execution time of the currently executing task, task identifier, task type, and task parameters. The execution time of the currently executing task refers to the time interval elapsed from the start of the task's execution on the executor to the current moment.

[0043] Status information refers to a set of data describing the executor's operating status, resource availability, and load characteristics, including but not limited to: queue information, resource utilization, and executor performance information. Queue information may include the current processing queue length, task data in the queue, task type, and task parameters. Resource utilization may include Central Processing Unit (CPU) utilization, memory utilization, and network input / output (IO) utilization. Actuator performance information may include executor identification and version information.

[0044] Historical information refers to information related to historical tasks that the executor has processed and completed before the current moment. Historical information includes the secondary task information for each of the multiple historical tasks, as well as the historical execution time, minimum execution time, maximum execution time, execution time distribution, and average execution time for each task type executed by the executor. Secondary task information includes, but is not limited to: the historical task's task type, task parameters, number of tasks, and task priority; the executor identifier processing the historical task; the actual execution time, start time, and end time of the historical task; the resource utilization rate during historical task processing; the queue length and number of concurrent tasks during historical task processing; the execution result of the historical task; and error information for the historical task. The historical task execution time refers to the duration taken by the executor to execute the historical task.

[0045] The actual execution time is later than or equal to the start time, and the end time is later than the actual execution time. The start time refers to the point in time when the historical task is scheduled to begin or is triggered; the actual execution time refers to the point in time when the historical task is beginning to occupy executor resources; and the end time refers to the point in time when all historical tasks have been processed. Execution results may include information indicating success or failure.

[0046] S204, for each actuator, candidate tasks are selected from multiple historical tasks based on the similarity between the first task information and the second task information.

[0047] In one embodiment, determining the similarity between first task information and second task information includes: combining multiple sub-information in the first task information to obtain a first feature vector; combining multiple sub-information in the second task information to obtain a second feature vector; calculating the similarity between the first feature vector and the second feature vector in at least two forms; and weighted summing of the similarities to obtain the similarity between the first task information and the second task information.

[0048] The first and second task information contain multiple sub-information items of the same type. For example, these sub-information items include task type (encoded as a numerical value), task parameters (such as data volume and processing type), and task priority.

[0049] The different forms of similarity can include cosine similarity, Euclidean distance, Manhattan distance, Minkowski distance, Chebyshev distance, and Jaccard similarity, etc.

[0050] For example, the similarity between the first task information and the second task information satisfies:

[0051]

[0052] in, Indicates the first task information. This indicates information about the second task. This indicates the similarity between the first task information and the second task information. The cosine similarity between the information from the first task and the information from the second task is represented. This represents the Euclidean distance between the first and second task information. This indicates the Jaccard similarity between the first and second task information. , and Indicates the weighting coefficient. .

[0053] In one embodiment, candidate tasks are selected from multiple historical tasks based on the similarity between the first task information and the second task information. This includes: sorting the multiple similarities in descending order to determine the top K similarities; and identifying the historical tasks corresponding to the K similarities as candidate tasks. For example, K is 50 or other values.

[0054] Alternatively, any form of similarity can be used to characterize the similarity between the first task information and the second task information. For example, any form of similarity can be any one of cosine similarity, Euclidean distance, Manhattan distance, Minkowski distance, Chebyshev distance, and Jaccard similarity.

[0055] S206, based on the second task information of each of the multiple candidate tasks and the task execution information of the executor, predict the estimated completion time of the executor's task.

[0056] S208. Based on the expected completion time and status information of the executor's task, obtain the task scheduling score of the executor.

[0057] The estimated task completion time refers to the total time it takes for the executor to complete the currently executing task and the waiting tasks in the processing queue. The higher the task scheduling score, the higher the probability that the executor will execute the target task.

[0058] S210, based on the task scheduling scores of each of the multiple executors, determines the scheduling executor from the multiple executors, and the scheduling executor is used to process the target task.

[0059] The scheduling executor is determined from among multiple executors based on their respective task scheduling scores. An example is provided below:

[0060] In one embodiment, determining the scheduling executor from multiple executors based on their respective task scheduling scores includes: determining the executor corresponding to the highest task scheduling score among the multiple task scheduling scores as the scheduling executor.

[0061] In one embodiment, determining the scheduling executor from multiple executors based on their respective task scheduling scores includes: determining the sum of the task scheduling scores of each of the multiple executors; for each executor, determining the ratio of the executor's task scheduling score to the sum of the scores as the executor's updated score; performing a weighted summation of the updated score and the executor's historical smoothed score to obtain the executor's current smoothed score; and determining the executor corresponding to the largest current smoothed score among the multiple current smoothed scores as the scheduling executor. Therefore, by smoothing the executor's task scheduling score using the sum of the scores and the historical smoothed score, switching caused by instantaneous fluctuations in the executor's task scheduling score can be avoided, thus improving the stability of the task scheduling process.

[0062] For example, the current smoothed score satisfies:

[0063]

[0064] in, This indicates the current smoothing score of the actuator. This indicates the updated score of the actuator. This represents the historical smoothing score of the actuator, which is the smoothing score of the actuator at time (t-1) before the current time t. This represents the smoothing coefficient, which can be any value from 0.2 to 0.5.

[0065] The method described in the above embodiments obtains first task information of the target task, as well as task execution information, status information, and historical information of multiple executors. The historical information includes second task information for each of the multiple historical tasks. For each executor, candidate tasks are selected from multiple historical tasks based on the similarity between the first and second task information. Thus, by introducing task similarity as a selection dimension, the task scheduling process can identify the executor's past ability and efficiency in handling similar tasks, providing historical experience for scheduling decisions. By predicting the executor's expected task completion time based on the second task information of multiple candidate tasks and the executor's task execution information, historical execution capabilities are correlated with real-time execution capabilities, achieving accurate prediction of future performance. By obtaining the executor's task scheduling score based on its expected completion time and status information, and then determining the scheduling executor from among the multiple executors based on their respective task scheduling scores, the scheduling executor is used to process the target task. Therefore, by ensuring that the target task is assigned to the most suitable and efficient executor, the accuracy and efficiency of task scheduling can be improved.

[0066] In one embodiment, the second task information includes the task execution history time, and the task execution information includes the execution time of the executor's currently executing task. For example... Figure 3 As shown, based on the second task information of each of the multiple candidate tasks and the task execution information of the executor, the estimated completion time of the executor's task is predicted, including the following steps:

[0067] S302, determine the historical average execution time among the historical execution times of multiple candidate tasks, and the first time difference between the historical average execution time and the executed time.

[0068] Specifically, the average historical execution time of multiple candidate tasks is determined as the historical average execution time.

[0069] S304, invoke the preset Bayesian execution time prediction model to predict the execution time of the currently executing task.

[0070] The task information for the currently executing task is described in the preceding text. The predicted execution time is the time predicted for the completion of the task currently being executed.

[0071] The pre-defined Bayesian execution time prediction model can be constructed based on the prior and posterior distributions, assuming that the task execution time follows a log-normal or gamma distribution.

[0072] When setting the prior distribution, it is assumed that the task execution time follows a log-normal distribution, satisfying:

[0073]

[0074] Alternatively, assume that the task execution time follows a gamma distribution, satisfying:

[0075]

[0076] Where T represents the task execution time. and Let be the parameters of the log-normal distribution, and be the prior mean and prior variance, respectively. This represents the model parameter vector. and represents the gamma distribution parameters.

[0077] In terms of parameters When making prior settings, parameters can be configured. Define a conjugate prior distribution, where:

[0078] mean The prior distribution satisfies:

[0079]

[0080] variance The prior distribution satisfies:

[0081]

[0082] in, This is the average of the historical task execution times corresponding to multiple training samples. This represents the variance derived from the historical task execution times corresponding to multiple training samples. Therefore, in subsequent actual predictions, , and Pre-set fixed values.

[0083] in, and based on and To obtain. For example, and They respectively satisfy:

[0084]

[0085]

[0086] Among them, A and B are preset values.

[0087] Given the task execution history times corresponding to multiple training samples, parameters can be calculated using Bayes' theorem. The posterior distribution satisfies:

[0088]

[0089] For a log-normal distribution, the mean The posterior distribution satisfies:

[0090]

[0091] in, and They respectively satisfy:

[0092]

[0093]

[0094] In subsequent actual predictions, These are parameters related to the actual task processing status and need to be calculated in real time. For example, in actual prediction, n represents the number of tasks that the executor has completed. This represents the execution time of the i-th task completed by the executor. It is expressed as the posterior standard deviation.

[0095] For the new task The pre-defined Bayesian execution time prediction model calculates the predicted distribution of its execution time, satisfying:

[0096]

[0097] Where D represents the task execution history time series, which consists of the task execution history time corresponding to multiple training samples, satisfying the following condition: Let i = 1, 2, ..., n. n represents the total number of historical execution times of the task.

[0098] Furthermore, the predicted expected execution time (log-normal distribution) satisfies:

[0099]

[0100] The predicted execution time P95 quantile satisfies:

[0101]

[0102] Therefore, based on the above analysis, in actual prediction, the expected value of the predicted execution time can be used as the output of the model prediction, that is, the predicted execution time.

[0103] Specifically, the preset Bayesian execution time prediction model is invoked to predict the predicted execution time of the current task, including: extracting the execution time of each task already completed by the executor from the task execution information of the executor; and obtaining the predicted execution time of the current task based on the number of tasks already completed by the executor, the execution time of each completed task, and the formula corresponding to the expected execution time.

[0104] Alternatively, in addition to the Bayesian execution time prediction model, other machine learning models such as Long Short-Term Memory (LSTM) and Extreme Gradient Boosting (XGBoost) can be used, as long as they can achieve the goal of accurately predicting task execution time.

[0105] S306, determine the second time difference between the predicted execution time and the executed time.

[0106] S308, determine the maximum value between the first time difference and the second time difference as the remaining execution time of the executor's task.

[0107] The remaining execution time refers to the time remaining before the executor can complete the currently executing task.

[0108] For example, the remaining time for task execution meets the following conditions:

[0109]

[0110] in, Indicates the remaining time for task execution. Indicates the historical average execution time. Indicates the execution time. This indicates the predicted execution time.

[0111] S310: For each waiting task in the executor's processing queue, the sum of the expected execution times for each waiting task is determined as the executor's waiting execution time.

[0112] The processing queue stores waiting tasks, i.e., tasks awaiting processing. The expected execution time refers to the time the executor will spend executing the waiting tasks.

[0113] Specifically, each waiting task corresponds to an expected execution time, and the sum of all expected execution times is determined as the executor's waiting execution time.

[0114] S312, the sum of the remaining execution time of the task and the waiting time is determined as the estimated completion time of the executor's task.

[0115] For example, the estimated completion time of the task meets the following requirements:

[0116]

[0117] in, Indicates the estimated completion time of the task. This indicates the waiting time for the task to be executed.

[0118] By employing the method described in the above embodiments and introducing a preset Bayesian execution time prediction model, it is possible to predict when the executor will complete the currently assigned tasks and waiting tasks. By making time predictions in advance, scheduling strategies can be optimized during task scheduling, thereby improving the accuracy and efficiency of task scheduling.

[0119] In one embodiment, such as Figure 4 As shown, the task scheduling score of the executor is obtained based on the executor's expected task completion time and status information, including the following steps:

[0120] S402, determine the average completion time among the expected completion times of the tasks of the multiple executors, and the average queue length among the queue lengths of the processing queues of the multiple executors.

[0121] The status information includes the queue length of the executor's processing queue.

[0122] S404: Based on the average completion time and the expected completion time of the executor's task, obtain the expected completion time score of the executor.

[0123] In one embodiment, a first ratio is determined between the negative of the expected completion time of the task and the average completion time; this first ratio is then processed using an exponential function to obtain the executor's expected completion time score. For example, the expected completion time score satisfies:

[0124]

[0125] in, The score represents the expected completion time. This indicates the average completion time.

[0126] S406: The load balancing score of the executor is obtained based on the average queue length and the queue length of the executor's processing queue.

[0127] The queue length of the processing queue refers to the number of tasks waiting to be assigned for execution in the processing queue.

[0128] For example, the load balancing score satisfies:

[0129]

[0130] in, This indicates the load balancing score. This indicates the queue length of the executor's processing queue. This represents the average queue length.

[0131] S408, based on the expected completion time score and the load balancing score, obtains the task scheduling score of the executor.

[0132] By employing the method described in the above embodiments, the task execution capabilities of the executor are considered from different dimensions to obtain the task scheduling score of the executor. Thus, dynamic task allocation based on the task scheduling scores of multiple executors can improve the accuracy of task scheduling.

[0133] The task scheduling score of the executor is obtained based on the expected completion time score and the load balancing score. This score can be determined in the following ways, for example:

[0134] In one embodiment, the task scheduling score of the executor is obtained based on the expected completion time score and the load balancing score, including: weighted summation of the expected completion time score and the load balancing score to obtain the task scheduling score of the executor. This helps improve resource utilization balance and overall system throughput, and can improve the accuracy of task scheduling in the system.

[0135] In one embodiment, status information includes resource utilization, and historical information includes the execution results of multiple historical tasks, with each task execution result indicating success or failure. Figure 5 As shown, the task scheduling score of the executor is obtained based on the expected completion time score and the load balancing score, including:

[0136] S502, determine the actuator's resource utilization score based on the resource utilization rate.

[0137] Specifically, resource utilization includes CPU utilization and memory utilization. A resource utilization score satisfies:

[0138]

[0139] in, This indicates the resource utilization score. Indicates CPU utilization. This indicates memory usage.

[0140] S504, based on the task execution results of multiple historical tasks, determine the task success rate of the target historical task among multiple historical tasks; the target historical task and the target task have the same task type.

[0141] Among them, the task success rate refers to the success rate of the target historical task that the executor successfully executes, that is, the proportion of the number of target historical tasks successfully executed by the executor to the total number of target historical tasks it receives.

[0142] S506 defines the historical reliability score of an actuator as the ratio of its task success rate to the average task success rate among the task success rates of multiple actuators.

[0143] For example, the historical reliability score satisfies:

[0144]

[0145] in, Indicates historical reliability score, Indicates the task success rate. This indicates the average task success rate.

[0146] S508 performs a weighted summation of the expected completion time score, load balancing score, resource utilization score, and historical reliability score to obtain the executor's task scheduling score.

[0147] For example, the task scheduling score satisfies:

[0148]

[0149] in, This indicates the task scheduling score. to These are the weighting coefficients.

[0150] By employing the method described in the above embodiments, the task execution capabilities of the executor are considered from different dimensions to obtain the task scheduling score of the executor. Thus, dynamic task allocation based on the task scheduling scores of multiple executors can improve the accuracy of task scheduling.

[0151] In one embodiment, the status information includes the queue length of the processing queue. The method further includes: identifying executors from a plurality of executors whose processing queue length is less than or equal to a preset length threshold as target executors; the number of target executors is at least two. In this case, for each executor, candidate tasks are selected from a plurality of historical tasks based on the similarity between first task information and second task information, including: for each target executor, candidate tasks are selected from a plurality of historical tasks based on the similarity between first task information and second task information. Therefore, through the executor selection mechanism based on queue length, it is possible to proactively avoid assigning new tasks to executors with severe backlogs, effectively preventing the spread of local overload and task queuing delays, helping to improve overall throughput and ensure the timeliness of task scheduling response.

[0152] In one embodiment, the method further includes: when the queue lengths of the processing queues of multiple executors are all greater than a preset length threshold, determining the executor with the shortest processing queue length from among the multiple executors as the scheduling executor. This enables locally optimal scheduling under high task load pressure, helps curb the chain reaction of queuing delays, and improves the overall throughput stability of the system.

[0153] Optionally, if the queue lengths of multiple executors are all greater than a preset length threshold, and the queue lengths of multiple executors are all the same, the executor with the lowest resource utilization rate is determined as the scheduling executor. This effectively avoids the problem of uneven resource utilization that may result from random allocation due to identical queue lengths, allowing for a more balanced consumption of system resources. While ensuring timely task processing, this improves the overall energy efficiency of cluster resource utilization.

[0154] Optionally, if the queue lengths of multiple executors' processing queues all exceed a preset length threshold, a random or round-robin strategy is used to determine the scheduling executor from among the multiple executors. Thus, by effectively preventing the risk of instantaneous overload that might be caused by concentrated selection of a single node, fairness of the system under task pressure can be achieved.

[0155] In one embodiment, the method further includes: when the target task is completed, obtaining task execution information of the target task, the task execution information including actual execution time and other task information; determining the time difference between the actual execution time and the predicted execution time of the target task as the prediction error; and using a Bayesian update rule to update the model parameters of the Bayesian execution time prediction model using other task information and the prediction error.

[0156] The method for determining the predicted execution time of the target task can be adapted to the description of how the predicted execution time of the currently executing task is determined. Other task information can be adapted to the description of the second task information.

[0157] For example, the prediction error satisfies:

[0158]

[0159] in, Indicates the prediction error. This indicates the actual execution time of the target task. This indicates the predicted execution time of the target task.

[0160] Update the model parameters using Bayesian update rules, satisfying:

[0161]

[0162] in, This indicates the task execution information for the target task.

[0163] Alternatively, online learning can be used to update model parameters in batches periodically (e.g., every 100 tasks) to avoid performance overhead caused by frequent updates.

[0164] In one embodiment, the method further includes: if the prediction error is greater than the product of the posterior standard deviation and a preset value, invoking executors other than the scheduling executor among multiple executors to participate in a new round of task scheduling. Thus, by assigning tasks to other executors and bypassing potentially faulty executors, the robustness and fault tolerance of the system in dynamic environments are improved.

[0165] Alternatively, if the prediction error exceeds the product of the posterior standard deviation and the preset value, the weight coefficient of the scheduler can be reduced. Thus, by directly reducing the scheduling weight of the scheduler, the probability of it being selected in the short term is decreased, preventing subsequent tasks from falling into the trap of inaccurate schedulers, thereby improving the system's robustness and fault tolerance in dynamic environments.

[0166] Alternatively, if the prediction error exceeds the product of the posterior standard deviation and the preset value, the weight coefficients of the executors other than the scheduling executor can be increased. This increases the weight coefficients of other executors, guiding task scheduling towards them and improving the system's task scheduling success rate.

[0167] The weight coefficient corresponding to the executor refers to the weight coefficient used when calculating the task scheduling score of the executor.

[0168] In one embodiment, if no heartbeat is received from the scheduler executor during the process of the scheduler executor processing the target task, or if the CPU or memory utilization of the scheduler executor continuously exceeds a preset threshold (e.g., 90%), then the executors other than the scheduler executor among the multiple executors are invoked to participate in a new round of task scheduling, or the weight coefficient corresponding to the scheduler executor is reduced, or the weight coefficient corresponding to the executors other than the scheduler executor among the multiple executors is increased.

[0169] Optionally, in the event of an anomaly in the scheduler executor, the anomaly information of the scheduler executor can be recorded for subsequent analysis, thereby improving the stability and reliability of the scheduler executor's task processing.

[0170] In summary, such as Figure 6As shown, an application scenario for a task scheduling method is provided. The task scheduling system 10 is configured with a data acquisition layer, a predictive analysis layer, and a decision execution layer. The data acquisition layer is used to collect task execution information, status information, and historical information from multiple executors. The predictive analysis layer is used to calculate the similarity between first task information and second task information, and to call a preset Bayesian execution time prediction model to predict task execution time. The decision execution layer is used to determine the task scheduling score of each executor, and based on the task scheduling scores of multiple executors, to determine the scheduled executor from among the multiple executors.

[0171] It is easy to understand that by expanding the routing policy enumeration, a predictive scheduling policy can be added; by integrating state acquisition, time prediction, score calculation, and executor selection functions, a predictive scheduling executor selector can be implemented; by acquiring the state information of all executors in parallel, an executor state collector can be implemented; and by using feature extraction, candidate task filtering, and Bayesian model prediction to complete execution time prediction, an execution time predictor can be implemented. These implementations ensure that this invention can be seamlessly integrated into existing scheduling frameworks, providing users with intelligent task scheduling services.

[0172] based on Figure 6 The structure shown is as follows: Figure 7 As shown, a task scheduling method is provided. Taking the application of this method to the scheduling component 102 in the task scheduling system 10 as an example, the method includes the following steps:

[0173] S702, obtain the first task information of the target task, as well as the task execution information, status information and historical information of each of the multiple executors; the historical information includes the task execution history time and task execution result of each of the multiple historical tasks, the task execution information includes the execution time of the currently executing task of the executor, and the status information includes the resource utilization rate.

[0174] S704, for each actuator, selects candidate tasks from multiple historical tasks based on the similarity between the first task information and the second task information.

[0175] S706, based on the second task information of each of the multiple candidate tasks and the task execution information of the executor, predicts the estimated completion time of the executor's task.

[0176] Specifically, the following steps are taken: First, determine the historical average execution time among multiple candidate tasks, and the first time difference between the historical average execution time and the actual execution time. Second, call a pre-defined Bayesian execution time prediction model to predict the current execution time of the task. Third, determine the second time difference between the predicted execution time and the actual execution time. Fourth, determine the maximum value between the first and second time differences as the remaining execution time of the executor. Fifth, for each waiting task in the executor's processing queue, determine the sum of the expected execution times of each waiting task as the executor's waiting execution time. Sixth, determine the sum of the remaining execution time and the waiting execution time as the executor's expected task completion time.

[0177] S708, determine the average completion time among the expected completion times of the tasks of multiple executors, and the average queue length among the queue lengths of the processing queues of the multiple executors.

[0178] S710 calculates the expected completion time score of the actuator based on the average completion time and the actuator's expected task completion time.

[0179] S712 calculates the load balancing score of the executor based on the average queue length and the queue length of the executor's processing queue.

[0180] S714 determines the actuator's resource utilization score based on resource utilization rate.

[0181] S716, based on the task execution results of multiple historical tasks, determine the task success rate of the target historical task among multiple historical tasks; the target historical task and the target task have the same task type.

[0182] S718 defines the historical reliability score of an actuator as the ratio of its task success rate to the average task success rate among multiple actuators.

[0183] The S720 performs a weighted summation of the expected completion time score, load balancing score, resource utilization score, and historical reliability score to obtain the executor's task scheduling score.

[0184] S722, among multiple executor task scheduling scores, the executor corresponding to the highest task scheduling score is determined as the scheduling executor, and the scheduling executor is used to process the target task.

[0185] The specific details of S702-S722 can be found in the aforementioned description and will not be repeated here.

[0186] The method described in the above embodiments improves the system's automation level and decision-making accuracy by predicting task execution time, eliminating the need for manual weight configuration and achieving intelligent scheduling. Prediction accuracy is enhanced through similar task matching and Bayesian prediction. Continuous learning and updating via the Bayesian method improves prediction accuracy over time, leading to continuous optimization of system performance. By comprehensively considering multiple factors such as queue status, historical execution time, task characteristics, and executor performance, scheduling decisions become more scientific and rational. By selecting the executor expected to complete earliest when all executors are busy, the average waiting time of tasks is significantly reduced. Therefore, the method provided in this application can perceive executor load changes in real time, dynamically adjust the scheduling strategy, and adapt to rapid changes in system status. Furthermore, this method can be used as an optional routing strategy for XXL-JOB without affecting existing functionality, allowing users to choose flexibly. Moreover, the prediction algorithm and scheduling strategy can be independently extended and optimized, supporting the future introduction of more advanced machine learning models.

[0187] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0188] Based on the same inventive concept, this application also provides a task scheduling apparatus for implementing the task scheduling method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more task scheduling apparatus embodiments provided below can be found in the limitations of the task scheduling method described above, and will not be repeated here.

[0189] In one exemplary embodiment, such as Figure 8 As shown, a task scheduling device is provided, which includes: an acquisition module 802, a filtering module 804, a processing module 806, an analysis module 808, and a determination module 810, wherein:

[0190] The acquisition module 802 is used to acquire first task information of the target task, as well as task execution information, status information and historical information of each of the multiple executors; the historical information includes second task information of each of the multiple historical tasks; the filtering module 804 is used to filter candidate tasks from the multiple historical tasks for each of the executors based on the similarity between the first task information and the second task information.

[0191] Processing module 806 is used to predict the expected completion time of the executor's task based on the second task information of each of the multiple candidate tasks and the task execution information of the executor; analysis module 808 is used to obtain the task scheduling score of the executor based on the expected completion time of the executor's task and the status information; determination module 810 is used to determine the scheduling executor from the multiple executors based on the task scheduling scores of each of the multiple executors, and the scheduling executor is used to process the target task.

[0192] In one embodiment, the second task information includes the historical execution time of the task, and the task execution information includes the execution time of the currently executing task of the executor; the processing module 806 is further configured to: determine the historical average execution time among the historical execution times of multiple candidate tasks, and a first time difference between the historical average execution time and the executed time; call a preset Bayesian execution time prediction model to predict the predicted execution time of the currently executing task; determine a second time difference between the predicted execution time and the executed time; determine the maximum value between the first time difference and the second time difference as the remaining task execution time of the executor; for each waiting task in the processing queue of the executor, determine the sum of the expected execution times corresponding to each waiting task as the waiting execution time of the executor; and determine the sum of the remaining task execution time and the waiting execution time as the expected completion time of the task of the executor.

[0193] In one embodiment, the analysis module 808 is further configured to: determine the average completion time among the expected completion times of the tasks of the plurality of executors, and the average queue length among the queue lengths of the respective processing queues of the plurality of executors; obtain an expected completion time score for the executor based on the average completion time and the expected completion time of the task of the executor; obtain a load balancing score for the executor based on the average queue length and the queue length of the processing queue of the executor; and obtain a task scheduling score for the executor based on the expected completion time score and the load balancing score.

[0194] In one embodiment, the status information includes resource utilization rate, and the historical information includes the task execution results of each of the plurality of historical tasks; the analysis module 808 is further configured to: determine the resource utilization rate score of the executor based on the resource utilization rate; determine the task success rate corresponding to the target historical task among the plurality of historical tasks based on the task execution results of each of the plurality of historical tasks; the target historical task and the target task have the same task type; determine the historical reliability score of the executor as the ratio of the task success rate to the average task success rate between the task success rates corresponding to the plurality of executors; and perform weighted summation processing on the expected completion time score, the load balancing score, the resource utilization rate score, and the historical reliability score to obtain the task scheduling score of the executor.

[0195] In one embodiment, the analysis module 808 is further configured to: determine a first ratio between the negative of the expected completion time of the task and the average completion time; process the first ratio using an exponential function to obtain the expected completion time score of the executor.

[0196] In one embodiment, the status information includes the queue length of the processing queue; the determining module 810 is further configured to: determine, from the plurality of executors, executors whose queue length of the processing queue is less than or equal to a preset length threshold as target executors; the number of target executors is at least two; the filtering module 804 is further configured to: for each target executor, filter candidate tasks from the plurality of historical tasks based on the similarity between the first task information and the second task information.

[0197] In one embodiment, the determining module 810 is further configured to: when the queue length of the processing queues of the plurality of executors is greater than the preset length threshold, determine the executor with the shortest queue length of the processing queue from the plurality of executors as the scheduling executor.

[0198] Each module in the aforementioned task scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0199] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data used in the task scheduling process. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a task scheduling method.

[0200] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0201] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0202] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0203] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0204] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0205] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0206] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0207] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A task scheduling method, characterized in that, The method includes: Obtain the first task information of the target task, as well as the task execution information, status information, and historical information of each of the multiple executors; the historical information includes the second task information of each of the multiple historical tasks. For each of the executors, candidate tasks are selected from a plurality of historical tasks based on the similarity between the first task information and the second task information; Based on the second task information of each of the multiple candidate tasks and the task execution information of the executor, the estimated completion time of the executor's task is predicted. Based on the expected completion time of the executor's task and the status information, the task scheduling score of the executor is obtained; Based on the task scheduling scores of the plurality of executors, a scheduling executor is determined from the plurality of executors, and the scheduling executor is used to process the target task; The second task information includes the task execution history time, and the task execution information includes the execution time of the currently executing task of the executor; the step of predicting the expected completion time of the executor's task based on the second task information of each of the multiple candidate tasks and the task execution information of the executor includes: The process involves: determining the historical average execution time among multiple candidate tasks, and a first time difference between the historical average execution time and the already executed time; invoking a preset Bayesian execution time prediction model to predict the predicted execution time of the currently executing task; determining a second time difference between the predicted execution time and the already executed time; determining the maximum value between the first time difference and the second time difference as the remaining execution time of the executor; for each waiting task in the executor's processing queue, determining the sum of the expected execution times of each waiting task as the executor's waiting execution time; and finally, determining the sum of the remaining execution time and the waiting execution time as the executor's estimated task completion time. The step of obtaining the task scheduling score of the executor based on the expected completion time of the executor's task and the status information includes: Determine the average completion time among the estimated task completion times of the plurality of executors, and the average queue length among the queue lengths of the respective processing queues of the plurality of executors; obtain the expected completion time score of the executor based on the average completion time and the estimated task completion time of the executor; obtain the load balancing score of the executor based on the average queue length and the queue length of the processing queue of the executor; obtain the task scheduling score of the executor based on the expected completion time score and the load balancing score.

2. The method according to claim 1, characterized in that, The status information includes resource utilization rate, and the historical information includes the task execution results of each of the multiple historical tasks. The step of obtaining the task scheduling score of the executor based on the expected completion time score and the load balancing score includes: Based on the resource utilization rate, determine the resource utilization rate score of the actuator; Based on the execution results of each of the multiple historical tasks, the success rate of the target historical task among the multiple historical tasks is determined; the target historical task and the target task have the same task type. The ratio of the task success rate to the average task success rate among the task success rates corresponding to the multiple executors is determined as the historical reliability score of the executor. The expected completion time score, the load balancing score, the resource utilization score, and the historical reliability score are weighted and summed to obtain the task scheduling score of the executor.

3. The method according to claim 1, characterized in that, The step of obtaining the expected completion time score of the executor based on the average completion time and the expected completion time of the executor's task includes: Determine a first ratio between the negative of the estimated completion time of the task and the average completion time; The first ratio is processed using an exponential function to obtain the expected completion time score of the actuator.

4. The method according to any one of claims 1 to 3, characterized in that, The status information includes the queue length of the processing queue; the method further includes: From the plurality of executors, the executors whose processing queue length is less than or equal to a preset length threshold are identified as target executors; the number of target executors is at least two. For each of the executors, the process of filtering candidate tasks from a plurality of historical tasks based on the similarity between the first task information and the second task information includes: For each target executor, candidate tasks are selected from multiple historical tasks based on the similarity between the first task information and the second task information.

5. The method according to claim 4, characterized in that, The method further includes: If the queue length of the processing queues of the plurality of executors is greater than the preset length threshold, the executor with the shortest processing queue length among the plurality of executors shall be determined as the scheduling executor.

6. A task scheduling device, characterized in that, The device includes: The acquisition module is used to acquire the first task information of the target task, as well as the task execution information, status information and historical information of each of the multiple executors; the historical information includes the second task information of each of the multiple historical tasks. A filtering module is used to filter candidate tasks from multiple historical tasks for each of the executors based on the similarity between the first task information and the second task information. The processing module is used to predict the estimated completion time of the executor's tasks based on the second task information of each of the multiple candidate tasks and the task execution information of the executor. The analysis module is used to obtain the task scheduling score of the executor based on the expected completion time of the executor's task and the status information; A determining module is configured to determine a scheduling executor from the plurality of executors based on the task scheduling scores of the plurality of executors, the scheduling executor being used to process the target task; The second task information includes the historical execution time of the task, and the task execution information includes the execution time of the currently executing task of the executor. The processing module is further configured to determine the historical average execution time among the historical execution times of multiple candidate tasks, and a first time difference between the historical average execution time and the executed time; call a preset Bayesian execution time prediction model to predict the predicted execution time of the currently executing task; determine a second time difference between the predicted execution time and the executed time; determine the maximum value between the first time difference and the second time difference as the remaining execution time of the executor; for each waiting task in the processing queue of the executor, determine the sum of the expected execution times corresponding to each waiting task as the waiting execution time of the executor; and determine the sum of the remaining execution time of the task and the waiting execution time as the expected completion time of the task of the executor. The analysis module is further configured to determine the average completion time among the expected completion times of the tasks of the plurality of executors, and the average queue length among the queue lengths of the respective processing queues of the plurality of executors; obtain the expected completion time score of the executor based on the average completion time and the expected completion time of the task of the executor; obtain the load balancing score of the executor based on the average queue length and the queue length of the processing queue of the executor; and obtain the task scheduling score of the executor based on the expected completion time score and the load balancing score.

7. The apparatus according to claim 6, characterized in that, The status information includes resource utilization rate, and the historical information includes the task execution results of each of the multiple historical tasks; the analysis module is further used for: Based on the resource utilization rate, determine the resource utilization rate score of the actuator; Based on the execution results of each of the multiple historical tasks, the success rate of the target historical task among the multiple historical tasks is determined. The target historical task is of the same type as the target task; The ratio of the task success rate to the average task success rate among the task success rates corresponding to the multiple executors is determined as the historical reliability score of the executor. The expected completion time score, the load balancing score, the resource utilization score, and the historical reliability score are weighted and summed to obtain the task scheduling score of the executor.

8. The apparatus according to claim 6, characterized in that, The analysis module is also used for: Determine a first ratio between the negative of the estimated completion time of the task and the average completion time; The first ratio is processed using an exponential function to obtain the expected completion time score of the actuator.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images