Task flow scheduling method and system based on decoupled task data model

Abstract

The invention discloses a task flow scheduling method and system based on a decoupling task data model. A single task data model participating in task flow scheduling is defined in a standardized manner, the communication messages and communication protocols of different tasks in a task flow are decoupled, and a globally unique distributed lock is defined for a task flow data model. In the task flow scheduling process, all scheduling nodes of the same task flow periodically compete for the distributed lock of the task flow, only the scheduling nodes winning the distributed lock are subjected to subsequent scheduling, subsequent tasks are obtained through a currently completed task to be scheduled, a protocol processor is used for converting task input and output data, and effective scheduling of the tasks is guaranteed through an idempotent mechanism, a task failure processing mechanism and the like. According to the method, the system design efficiency and the task flow execution stability can be improved, and a task is effectively prevented from being terminated due to accidental factors such as downtime of a single machine.

Description

technical field [0001] The invention belongs to the field of computer software, and in particular relates to a task flow scheduling method and system. Background technique [0002] Task flow refers to a series of task sequences with sequential dependencies, and the execution output results of the former in the sequence are used as input parameters to drive the execution of the latter. Multiple tasks in a task flow often run in a heterogeneous system environment, so when defining a specific task flow, it is necessary to customize and develop each task in the task flow. The execution of the task flow adopts a chain drive, and the former notifies the latter to continue execution after the execution is completed. [0003] The life cycle of a task flow is usually: first, perform custom development for each task in the task flow according to requirements; then deploy the task to a specific target computing node according to the system environment on which each task depends; and t...

AI Technical Summary

Problems solved by technology

[0004] The existing task flow scheduling process has the following shortcomings: 1. There are unstable factors in the network, and there is no unified task-driven mechanism to ensure that tasks can reliably execute chain tasks in an unreliable network

3. In the task scheme executed by the chain drive, the message content of the previous task will be strongly coupled with the message content of the next task, and the former needs to be compatible with the communication protocol of the latter. Each task of the task flow in a heterogeneous system may adopt a different communication protocol, resulting in a protocol explosion

4. For the sake of high availability of the system, each task in the task flow needs to implement task persistence and support regular recall, and the above functions are not related to the task itself, which is duplication of labor

5. There is no unified management perspective to show the execution progress of the current task flow and possible failure reasons

6. The unreliability of network message transmission leads to the possibility that the same computing task may be executed multiple times

Images