Distributed task management platform and method capable of supporting global multi-machine room deployment

Abstract

The invention discloses a distributed task management platform and method capable of supporting the global multi-machine room deployment. The distributed task management platform comprises a task executor, a task scheduler, a registration center, a task management server and a task control foreground, wherein the task executor specifically executes a batch processing task, runs in an application service process and is provided with a plurality of nodes, and in the process of executing the batch processing task, the task execution state of the task executor is reported to the task management server through a message; the task scheduler executes fragmentation scheduling of the tasks; the registration center is responsible for recording and notifying online and offline states of the task executor nodes, triggering fragmentation scheduling and storing various information; the task management service is responsible for task information management and manually triggering task execution; andthe task control foreground is deployed in a central machine room, and the task management server of each machine room is called to manage the task of each machine room, so that single-console multi-machine-room distributed task management is realized. The method has the beneficial effect that the load of the client node is reduced.

Description

technical field [0001] The present invention relates to the technical field related to the Internet, in particular to a distributed task management platform and method supporting global deployment of multiple computer rooms. Background technique [0002] In the business system, there is often a need for scheduled batch processing, similar to the function of Linux's Crontab, these batch processing tasks are more dependent on the data of the business system, and for scenarios with strong data consistency, the execution status of the task needs to be Real-time feedback and presentation, while ensuring high availability and failover of tasks. Therefore, only relying on the Crontab of the Linux operating system is difficult to meet the needs. At this time, it is necessary to choose a unified distributed task management platform to solve these problems. In order to improve the parallelism of batch processing tasks, a task will be decomposed into multiple subtasks (shards) for exe...

AI Technical Summary

Problems solved by technology

[0003] In open source software, there are some such frameworks, the most famous of which is Dangdang’s open source elastic-job-lite. The current problem is that the middleware related to open source distributed task management is difficult to meet the single platform management global Scheduling requirements for multi-computer room deployment tasks. For multi-computer room requirements, only a set of task management systems can be independently deployed in each cluster, which brings complexity to global multi-computer room distributed task management, monitoring, and task rule delivery.

At present, the main open source solution elastic-job-lite implements job time scheduling, slice scheduling, job execution, log recording, etc. in the form of a rich client, which leads to excessive load on the client, which greatly affects the stability and reliability of the business system. Scalability

Images