Task scheduling method and device

Abstract

The embodiment of the invention provides a task scheduling method and device. The method comprises the steps that a task scheduling request is acquired; the task scheduling request carries a task pathcorresponding to a to-be-executed task; according to the task path corresponding to the to-be-executed task and task configuration information, a result output path and an execution mode corresponding to the to-be-executed task are determined; the task path corresponding to the to-be-executed task, the result output path and the execution mode are correspondingly issued to task execution equipment for executing a task; after it is determined that the to-be-executed task is successfully executed, the result output path corresponding to the to-be-executed task serves as the task path corresponding to the to-be-executed task again; the above process is repeated until the cycle scheduling and execution of the task is completed. The task scheduling method and device have the advantages that the maintenance cost of the task scheduling can be reduced, and the task execution efficiency is improved.

Description

technical field [0001] The present application relates to the field of computer equipment, in particular to a task scheduling method and device. Background technique [0002] In big data processing, the efficiency of task scheduling directly affects the efficiency of big data processing, and also affects the ability of the data platform to execute tasks submitted by users. [0003] At present, there are two main ways of task scheduling, one is based on DAG graph (Directed Acyclic Graph, non-loop directed graph) for task scheduling, for example, divide a job into multiple tasks, and create a DAG graph according to the dependencies between tasks , and perform task scheduling according to the DAG graph. The other is a scheduled task scheduling mechanism, specifically, configure a time interval, schedule tasks according to the configured time interval and trigger task execution. [0004] In the above-mentioned first method, a complex DAG graph needs to be created in advance be...

AI Technical Summary

Problems solved by technology

[0004] In the above first method, it is necessary to pre-create a complex DAG graph before task scheduling, and if the scheduling order changes during the task scheduling process, the DAG graph needs to be modified, so this method has the problem of high maintenance cost of task scheduling

In the above-mentioned second method, when scheduling tasks according to the preset time interval and triggering task execution, if there is data delay and the data has not been transmitted to the task execution module, it will lead to waste of task scheduling and execution, so this method Has the problem of low task execution efficiency

Images