Method and device for distributed workflow dispatching

Abstract

The invention provides a method and a device for distributed workflow dispatching. The method comprises the following steps: acquiring an operation execution instruction supplied by a user, a workflow topological relation and an inter-operation dependency; searching for a node corresponding to the operation execution instruction on the basis of the workflow topological relation and the inter-operation dependency; acquiring a prepared node; confirming a resource quota required by the prepared node by a resource management module; adopting a colony resource scheduling system for scheduling the resource corresponding to a colony resource quota to the resource management module; sending a starting instruction by the resource management module which has acquired the resource through a resource scheduling agent, thereby starting a remote operation executing module for executing the operation execution instruction. According to the invention, the topology for traversing Flow on the basis of the complex inter-operation dependency is adopted for realizing the distributed running of the workflow, the scheduling logic and the executing logic in the workflow are separated and the workflow scheduling and the resource management logic are separated.

Description

technical field [0001] The invention relates to the technical field of computer networks, in particular to a distributed workflow scheduling technology. Background technique [0002] In the existing workflow management system, all workflow jobs are executed in the form of multi-threading on one host. The disadvantage is that when faced with a large number of large-scale workflow execution requests, the host's single machine resources (CPU, memory and Due to the limited size of disk), it is unable to respond to requests in a timely manner, that is, it does not have good scalability in terms of job processing scale and resource level; and in reality, the dependencies between workflow jobs are not only sequential dependencies, but often include more complex Dependencies, and the existing workflow scheduling system cannot separate the scheduling and execution logic in the workflow, as well as the scheduling and resource management logic in the workflow. [0003] Therefore, how ...

AI Technical Summary

Problems solved by technology

[0002] In the existing workflow management system, all workflow jobs are executed in the form of multi-threads on one host. The disadvantage is that when faced with a large number of large-scale workflow execution requests, the host's single machine resources (CPU, memory and Due to the limited size of disk), it is unable to respond to requests in a timely manner, that is, it does not have good scalability in terms of job processing scale and resource level; and in reality, the dependencies between workflow jobs are not only sequential dependencies, but often include more complex dependencies, and the existing workflow scheduling system cannot separate the scheduling and execution logic in the workflow, as well as the scheduling and resource management logic in the workflow

Images