Workflow task scheduling method under distributed environment

Abstract

The invention discloses a workflow task scheduling method under the distributed computing environment. A directed acyclic graph is used for describing workflow tasks and resources, the average task execution time and the average task transmission time are calculated on the basis of node weight information and edge weight information so as to judge the type of a workflow task and perform classified model processing, the task with the data dependency is cut into a plurality of independent task sets, each task set contains one or more tasks, the multiple tasks in each task set have the data dependency, and therefore the workflow tasks are converted into independent tasks. The task sets gather the tasks with the large data size or the large data transmission quantity together, meanwhile, the number of tasks needing to be independently distributed is decreased, and the post-task scheduling performance is improved on the whole. In addition, when the number of workflow tasks is increased, the gathering operation of the tasks just needs to be enhanced, all tasks do not need to be processed any more, and the better expansibility is achieved.

Description

technical field [0001] The invention belongs to workflow task scheduling technology and workflow task modeling technology under distributed computing, in particular to a workflow task scheduling method under distributed computing environment. Background technique [0002] Task scheduling is the most important part in a distributed computing environment, and it also plays an irreplaceable role in the entire distributed infrastructure. Task scheduling in a distributed computing environment requires finding the best solution for task resource allocation in consideration of time, cost, reliability, availability, throughput, and resource utilization. [0003] Task scheduling in a distributed computing environment is mainly divided into independent task scheduling and workflow task scheduling. A workflow task contains several tasks, and these several tasks have data dependencies. Existing methods aim to reduce the cost of task execution when scheduling workflow tasks. For exampl...

AI Technical Summary

Problems solved by technology

However, none of the above algorithms considers the impact of the data dependencies between workflow tasks on the task scheduling effect, resulting in high communication costs for workflow tasks, long execution completion time, and low efficiency of workflow task scheduling.

Images