Task execution and scheduling method, system and device, computing equipment and medium

Abstract

The invention provides a task execution and scheduling method, system and device, computing equipment and a medium, and the method comprises the steps: obtaining a task instance corresponding to a preset function sent by a scheduler, and configuring an execution environment of the task instance; determining at least one to-be-executed step included in a to-be-executed task corresponding to the task instance based on the task instance; generating an execution entity corresponding to each to-be-executed step in the at least one to-be-executed step so as to execute the corresponding to-be-executed step by utilizing the execution entity; or, when the preset execution time of the to-be-executed task with the first execution sequence corresponding to the preset function is reached, determining other to-be-executed tasks having a first dependency relationship with the to-be-executed task; generating a task instance of each to-be-executed task based on the feature information of the to-be-executed task and other to-be-executed tasks; and generating a first instance graph based on the first dependency relationship and the task instance, and sending the task instance to a corresponding executor based on the first instance graph.

Description

technical field [0001] The present disclosure relates to the field of computer technology, and in particular, to a task execution and scheduling method, system, device, computing device and medium. Background technique [0002] With the advent of the Internet era, the amount of data has increased dramatically, and big data tools have been used to replace traditional tools in classic data warehouses. A complete big data analysis system must be composed of many task units. All task units and their dependencies form a complex workflow. In order to manage complex workflows, various task scheduling systems have emerged. [0003] At present, a common task scheduling system is to combine tasks developed by different users into a task group, in which each task in the task group has certain execution dependencies, and based on the execution dependencies, they can cooperate with each other to achieve certain preset functions , when scheduling the tasks in the task group, only when th...

AI Technical Summary

Problems solved by technology

[0004] However, when using the above method to schedule each task in the task group, if the execution time of a certain task in the task group is reached, but the task is not started, it is necessary to check in turn whether the upstream tasks it depends on are Execution errors, when there are many tasks between the wrong upstream task and this task, the method of sequential inspection will increase the difficulty of operation and maintenance and reduce the performance of the task scheduling system

Images