Task scheduling method and device and medium

Abstract

The invention discloses a task scheduling method and device and a medium, and is suitable for the technical field of data processing. A task is divided into a plurality of standard to-be-scheduled tasks and divided into a plurality of execution windows at the same time, the current standard to-be-scheduled task is executed in one execution window, and then the next standard to-be-scheduled task is executed, namely, the scheduling tasks are executed in a serial mode. The waiting time is recorded during scheduling processing so as to timely sense the pressure of task execution to the downstream, and if the waiting time reaches the maximum execution time, the downstream pressure is relatively high; abandoning the current standard to-be-scheduled task, and dynamically adjusting the scheduling configuration parameters at the same time so as to obtain the next standard to-be-scheduled task in the subsequent current execution window. The problems that the downstream is overloaded and the server cannot work normally under the normal condition are avoided, and the stability of concurrent scheduling of large-batch tasks is improved.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a task scheduling method, device and medium. Background technique [0002] During data processing, the input data generally contains multiple items, and there is no human interaction in the processing process. A computer program executes a series of tasks based on a batch of input to realize the scheduling and processing of large batches of tasks. [0003] Under normal circumstances, large batches of tasks are generally dispatched at one time (upstream system) to threads for asynchronous processing (downstream system), and the results returned by the downstream system may be scheduled to timeout and cause downstream overload, and the upstream system does not know that the call results are still in accordance with the The previous rate and concurrent requests to the downstream system will cause a vicious circle, which may lead to an "avalanche effect"; if the retur...

AI Technical Summary

Problems solved by technology

[0003] Under normal circumstances, large batches of tasks are generally dispatched at one time (upstream system) to threads for asynchronous processing (downstream system), and the results returned by the downstream system may be scheduled to timeout and cause downstream overload, and the upstream system does not know that the call result is still in accordance with the The previous rate and concurrent requests to the downstream system will cause a vicious circle, which may lead to an "avalanche effect"; if the returned result has a scheduling interruption, it will affect the failure of subsequent tasks to return; the above two situations result in poor stability of the scheduling task , causing the server to fail to work properly or even crash

Images