Planned task execution method and system based on batch scheduling center

Abstract

The invention discloses a method and system for executing a planned task based on a batch scheduling center. The method for executing a planned task includes: step S1: analyzing the relationship between each batch of tasks, connecting each batch of tasks in sequence, and generating a A task execution flow chart with a fixed direction; step S2: according to the order of each node in the task execution flow chart, execute and complete all the tasks in the task execution flow chart; wherein, during the task execution process, in the task execution flow chart in different colors Identify the execution status of each batch of tasks in real time, and allow users to operate each batch of tasks to monitor the task execution process; each batch of tasks corresponds to at least one task partition, and each task partition pulls up a corresponding task partition by calling the DFS application service The parameters of each pod container are uniformly configured by the batch scheduling center so that each pod container has its own independent resource space.

Description

technical field [0001] The invention relates to a batch scheduling center-based plan task execution method and system. Background technique [0002] There are the following defects when the existing batch scheduling center application server pulls up the batch business application server process: [0003] 1. Batch execution does not have a complete execution planning process, and there is no unified module to manage the relationship and execution sequence between tasks. Every time a batch is executed, it is necessary to manually determine whether it can be executed. [0004] 2. DFS (Data Flow Server) calls the local jar package to start the batch task process instead of the container pod. The process starts directly on the virtual machine or physical machine, which is not enough to monitor whether the process is successfully started and whether there is an error in operation. , the startup parameters of the process, the startup process, etc., the resources are not isolated ...

AI Technical Summary

Problems solved by technology

[0003] 1. There is no complete execution plan process for batch execution, and there is no unified module to manage the relationship and execution sequence between tasks. Every time a batch is executed, it is necessary to manually determine whether it can be executed

[0004] 2. DFS (Data Flow Server) calls the local jar package to start the batch task process instead of the container pod. The process starts directly on the virtual machine or physical machine, which is not enough to monitor whether the process is successfully started and whether there is an error in operation. , the startup parameters of the process, the startup process, etc., the resources are not isolated when the batch application process is started, and the batches will preempt resources from each other, so that the resource allocation is uneven

[0005] 3. The pre-loading method cannot be used when executing the task chain in the plan. The subsequent task execution pulls up the container pod and must wait for the previous task to execute successfully before operating. The disadvantage of this is that when the previous task is successfully executed, the subsequent task is not executed immediately. , but it takes a lot of time to pull up container pods and start batch applications

[0006] 4. DFS needs to deploy application servers separately, which increases resource costs and maintenance costs; when the batch scheduling center executes tasks, it must request access to the DFS application server through the scheduling center, and then DFS will pull up the batch execution process to increase the scheduling center's access to DFS Network requests between application servers

Images