Scheduled task execution method and system based on batch scheduling center

Abstract

The invention discloses a scheduled task execution method and system based on a batch scheduling center, and the method comprises the steps: S1, analyzing the relation between all batches of tasks, enabling all batches of tasks to be connected in sequence to generate a task execution flow chart in a fixed direction; S2, sequentially executing all tasks for completing the task execution flow chartaccording to the sequence of each node of the task execution flow chart; wherein in the task execution process, the execution state of each batch of tasks is identified in real time with different colors in the task execution flow chart, and a user is allowed to operate each batch of tasks so as to monitor the task execution process; wherein each batch task corresponds to at least one task partition, each task partition pulls up a corresponding pod container by calling a DFS application service, and parameters of each pod container are uniformly configured by the batch scheduling center, so that each pod container has a respective independent resource space.

Description

technical field [0001] The invention relates to a batch scheduling center-based plan task execution method and system. Background technique [0002] There are the following defects when the existing batch scheduling center application server pulls up the batch business application server process: [0003] 1. Batch execution does not have a complete execution planning process, and there is no unified module to manage the relationship and execution sequence between tasks. Every time a batch is executed, it is necessary to manually determine whether it can be executed. [0004] 2. DFS (Data Flow Server) calls the local jar package to start the batch task process instead of the container pod. The process starts directly on the virtual machine or physical machine, which is not enough to monitor whether the process is successfully started and whether there is an error in operation. , the startup parameters of the process, the startup process, etc., the resources are not isolate...

AI Technical Summary

Problems solved by technology

[0003] 1. There is no complete execution plan process for batch execution, and there is no unified module to manage the relationship and execution sequence between tasks. Every time a batch is executed, it is necessary to manually determine whether it can be executed

[0004] 2. DFS (Data Flow Server) calls the local jar package to start the batch task process instead of the container pod. The process starts directly on the virtual machine or physical machine, which is not enough to monitor whether the process is successfully started and whether there is an error in operation. , the startup parameters of the process, the startup process, etc., the resources are not isolated when the batch application process is started, and the batches will preempt resources from each other, so that the resource allocation is uneven

[0005] 3. The pre-loading method cannot be used when executing the task chain in the plan. The subsequent task execution pulls up the container pod and must wait for the previous task to execute successfully before operating. The disadvantage of this is that when the previous task is successfully executed, the subsequent task is not executed immediately. , but it takes a lot of time to pull up container pods and start batch applications

[0006] 4. DFS needs to deploy application servers separately, which increases resource costs and maintenance costs; when the batch scheduling center executes tasks, it must request access to the DFS application server through the scheduling center, and then DFS will pull up the batch execution process to increase the scheduling center's access to DFS Network requests between application servers

Images