Decentralized distributed timed task processing method

Abstract

The invention discloses a decentralized distributed timed task processing method, belongs to the technical field of data processing. The following problems are solved that: 1) hardware cost and maintenance cost are high; 2) the lightweight tasks and the heavy tasks are not isolated, so that the tasks are blocked; 3) task scheduling strongly depends on a third-party component; and 4) the problem that a preemptive routing strategy has problems and a large probability is concentrated to one task machine to execute tasks. The technical scheme of the invention comprises the following steps of: 1, configuring task dynamic information and basic parameters; 2, after the task machine is started, automatically registering the task; and 3, after registration is completed, the task machine putting the task into a corresponding task pool for execution, and storing an execution result into a database, so that timed task business scenes of most systems can be met.

Description

technical field [0001] The invention belongs to the technical field of data processing, and in particular relates to a decentralized distributed timing task processing method. Background technique [0002] In most business systems, we usually denoise, process, and summarize some raw data regularly to make reports or provide effective data support for complex businesses. With the development of business and the continuous precipitation of data, we will need more and more timed tasks, and the amount of data processed by a single timed task will also increase, and the corresponding processing time will also become longer and longer. Therefore, in order not to affect the smooth use of the business system in the later stage, the scheduled tasks are often separated from the business system and made into separate task services to achieve the effect of decoupling. [0003] The existing technical solutions are open source distributed scheduling task frameworks xxl-job, tbschedule, e...

AI Technical Summary

Problems solved by technology

[0010] 1) To ensure the high availability of the cluster, the scheduling center needs to deploy at least two active and standby nodes. For small and medium business systems, the hardware cost and maintenance cost are high;

[0011] 2) Lightweight tasks are not isolated from heavyweight tasks, which will lead to task congestion;

[0012] 3) Task scheduling strongly depends on third-party components (such as zookeeper);

[0013] 4) There is a problem with the preemptive routing strategy, and there is a high probability that a task machine will be concentrated to perform tasks

Images