Task scheduling method and device based on distributed scheduling framework

Abstract

The invention relates to the technical field of big data, in particular to a task scheduling method and device based on a distributed scheduling framework. The method comprises the steps of receivinga task scheduling request, wherein the task scheduling request carries a to-be-scheduled task identifier; obtaining a task size of a to-be-scheduled task corresponding to the to-be-scheduled task identifier; fragmenting the to-be-scheduled task according to the task size to obtain fragmented tasks, and obtaining the priority of each fragmented task according to preset logic; calculating a task load rate of each execution node in the distributed scheduling framework; and allocating each fragmentation task to each execution node according to the task load rate and the priority so as to indicateeach execution node to perform task scheduling on the allocated fragmentation task. By adopting the method, the task scheduling efficiency can be improved. In addition, the invention also relates to ablockchain technology, and the working state of each execution node is stored in the blockchain.

Description

technical field [0001] The present application relates to the technical field of big data, in particular to a task scheduling method and device based on a distributed scheduling framework. Background technique [0002] In the field of database asynchronous scheduling, the asynchronous scheduling framework supports asynchronous scheduling, and time expressions can be set for scheduling. When there is no task, it will be scheduled when the time is met, thus wasting system computing resources. [0003] In traditional technology, task scheduling can also be performed in a distributed environment, but in the process of task scheduling, it is also necessary to control the same task to run on a single node. Such a computing method will seriously waste the computing power of the distributed environment. Contents of the invention [0004] Based on this, it is necessary to provide a task scheduling method and device based on a distributed scheduling framework that can improve task s...

AI Technical Summary

Problems solved by technology

[0002] In the field of database asynchronous scheduling, the asynchronous scheduling framework supports asynchronous scheduling, and time expressions can be set for scheduling. When there are no tasks, they will be scheduled when the time is met, thus wasting system computing resources.

[0003] In traditional technology, task scheduling can also be performed in a distributed environment, but in the process of task scheduling, it is also necessary to control the same task to run on a single node. Such a computing method will seriously waste the computing power of the distributed environment.

Images