Operation scheduling method and operation scheduler

Abstract

The application relates to an operation scheduling method and an operation scheduler. The operation scheduling method comprises the following steps: receiving operation scheduling request information which carries the identification information of an operation; inquiring an empirical data database and confirming that whether the historical operation information of the operation exists in the empirical data database or not, and judging the type of the operation; determining the execution cost group information of the operation according to the type of the operation; determining the scheduling algorithm to be used for scheduling the operation; generating the scheduling plan data of the operation through the execution cost group information and all the available resource group information of a scheduling system by virtue of the scheduling algorithm to complete the operation scheduling. The operation scheduling method achieves the purpose of dynamically selecting a scheduling algorithm to make a scheduling plan according to preset scheduling algorithms, so that the utilization rate of system resources is improved.

Description

technical field [0001] The present application relates to the field of job scheduling, in particular to a job scheduling method and a scheduler thereof. Background technique [0002] The current ETL (English full name: Extract-Transform-Load, Chinese: Extraction, Transformation, Loading) workflow scheduling methods all use the most basic FIFO (English full name: First In First Out, Chinese: First Come First Service) scheduling mode or A separate algorithm is scheduled. Due to the inconsistency of scheduled job types and requirements, the operating environment is more complicated. For example, the data engine, running cycle, QoS requirements, and resource differences required by the job are complex and changeable. And with the advent of the cloud computing era, the number and complexity of job scheduling will increase dramatically. [0003] Existing systems or common systems often use fixed strategies to schedule jobs, and different strategies and algorithms have very diff...

AI Technical Summary

Problems solved by technology

[0005] Existing job scheduling methods have the following disadvantages: On the one hand, existing scheduling methods use fixed strategies or algorithms, and cannot dynamically select appropriate strategies or algorithms to formulate job execution plans according to system resource conditions, which affects system resources. On the other hand, the existing scheduling method does not make good use of historical data to estimate the execution cost group information of the job, so the estimated execution cost group information cannot be used as an important factor for making a job execution plan, which in turn affects the utilization of system resources

Images