Batch processing task scheduling method and system based on general workflow

Abstract

The invention relates to a batch processing task scheduling method and system based on general workflow, and the method comprises the steps: constructing a task model which comprises a task package, a task group, a task layer, a task and a job; obtaining task configuration, analyzing the task configuration into a task group, and judging whether the task group can be executed or not; the task group is decomposed to obtain a task layer, the task layer is configured with a plurality of tasks, whether the tasks can be executed is judged, and the tasks of the task layer are executed; and entering an execution process of the job, asynchronously initiating job execution, synchronously polling and querying a job execution state, and performing an execution process of the next job when a job query result is successful. The invention provides a set of general workflow mechanism, hierarchical management of task scheduling is realized through hierarchical classification of task models, efficient utilization of task scheduling resources is realized based on containerized batch processing components, and the problem of efficient development of task scheduling in a scene that a large number of batch processing tasks are mutually dependent is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of batch processing task scheduling, and in particular to a batch processing task scheduling method and system based on general workflow. Background Art

[0002] The existing batch scheduling technologies mainly focus on solving the configuration of scheduling rules, efficient use of scheduling resources, and fault-tolerant processing of scheduling tasks. There is no complete set of technologies that can be used to solve the scheduling of batch tasks in complex field task dependency scenarios. The existing task scheduling can be roughly divided into the following categories:

[0003] 1. Scheduled task solution, which solves the task dependency problem by setting different trigger times for interdependent tasks;

[0004] 2. Parent-child task solution: by setting a parent task for a task, the child task can only be executed after the parent task is successfully executed;

[0005] 3. Directed acyclic graph solution: The online editing and publishing of directed acyclic graphs are realized through DAG visual editing tools, and the dependencies between tasks are described in the graph.

[0006] The existing task scheduling method has some defects:

[0007] 1. The implementation of the scheduled task solution is relatively simple. The dependency relationship between the upper and lower tasks can be indirectly realized through the time sequence. However, due to the unpredictability of the execution state of each task, in special cases, the upstream task is not executed due to the large business processing volume, but the downstream task reaches the execution time point, resulting in the downstream task being executed after the upstream task has not been completed, resulting in task scheduling disorder;

[0008] 2. The parent-child task solution can only solve the task dependency relationship in the scenario of one upstream task and multiple downstream tasks, because downstream tasks cannot be configured with multiple parent tasks, resulting in a single dependency relationship;

[0009] 3. Directed acyclic graph solution: This solution needs to rely on complex directed graph editing tools on the front end. Each task scheduling requires the development of a separate task scheduling flowchart, which has relatively high technical requirements for developers. Summary of the invention

[0010] In order to address the shortcomings of the prior art, the present invention proposes a universal workflow mechanism, which implements hierarchical management of task scheduling through hierarchical classification of task models, breaks down an original large task that is time-consuming to run into different small tasks and small jobs, and implements efficient utilization of task scheduling resources based on containerized batch processing components, solving the problem of efficient development of task scheduling in scenarios where a large number of batch processing tasks are interdependent.

[0011] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0012] A batch task scheduling method based on a general workflow, characterized by comprising:

[0013] S1. Constructing a task model, wherein the task model includes a task package, a task group, a task layer, a task, and a job;

[0014] S2. Get the task configuration, parse the task configuration into a task group, call the task group judgment process in a loop, determine whether the task group can be executed, and enter the execution process of the task group;

[0015] S3. Decomposing the task group to obtain a task layer, the task layer is configured with multiple tasks, determining whether the task can be executed, entering the task execution process, and executing the task layer tasks;

[0016] S4. Enter the job execution process, asynchronously initiate job execution, synchronously poll and query the job execution status, and when the job query result is successful, proceed to the next job execution process.

[0017] Furthermore, the task package is used to distinguish the classification management of task processes of different businesses, the task layer is used to describe the dependency relationship between tasks, and the task is used to configure task attributes, which include task type, whether to postpone, and the number of task slices.

[0018] Furthermore, the cyclically calling task group judgment process to judge whether the task group can be executed and entering the execution process of the task group includes:

[0019] Calling the execution day of the task group, and judging whether the task group can be executed on that day according to the scheduling rule of the task group, wherein the scheduling rule includes a specified execution day and execution frequency, and the scheduling rule is configurable;

[0020] If the current day is not the execution day of the task group, the execution process of the current task group is terminated and the next task group is polled;

[0021] When the current day is the execution day of the task group, the execution process of the current task group is entered.

[0022] Furthermore, the execution process of the task group includes:

[0023] The execution process of the task group decomposes the task group into task layers, sorts the task layers according to their priorities, and executes the task layers with high priorities first;

[0024] Concurrently execute tasks at the same task layer and synchronously query task execution results;

[0025] When all tasks are executed successfully, enter the next task layer.

[0026] Furthermore, the synchronous query task execution result includes:

[0027] Synchronous polling to query the task execution result, the polling frequency is consistent with the frequency of judging whether the task can be executed;

[0028] When the task execution result is successful, query the execution result of the next task in the same task layer; when all the tasks in the same task layer have successful execution results, execute the next task layer;

[0029] When the task execution result is failure, the execution process of the task is blocked.

[0030] Further, the determining whether the task is executable includes:

[0031] Determining whether the task is executable according to the task configuration;

[0032] When a task is postponed, wait for the task to be executed according to the postponement time.

[0033] The present invention also relates to a batch processing task scheduling system based on a general workflow, characterized in that it comprises:

[0034] A model building module, used to build a task model, wherein the task model includes a task package, a task group, a task layer, a task, and an operation;

[0035] A configuration parsing module is used to obtain task configuration, parse the task configuration into a task group, cyclically call the task group judgment process, judge whether the task group can be executed, and enter the execution process of the task group;

[0036] A task execution module is used to decompose the task group to obtain a task layer, wherein the task layer is configured with multiple tasks, determine whether the task can be executed, enter the task execution process, and execute the task of the task layer;

[0037] The job execution module is used to enter the job execution process, asynchronously initiate job execution, synchronously poll and query the job execution status, and proceed to the next job execution process when the job query result is successful.

[0038] The present invention also relates to a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and the computer program implements the above method when executed by a processor.

[0039] The present invention also relates to an electronic device, characterized in that it comprises a processor and a memory;

[0040] The memory is used to store program instructions;

[0041] The processor is used to execute the above method by calling program instructions.

[0042] The present invention also relates to a computer program product, comprising a computer program and / or instructions, characterized in that the computer program and / or instructions implement the steps of the above method when executed by a processor.

[0043] The beneficial effects of the present invention are:

[0044] The batch task scheduling method based on a general workflow described in the present invention can solve the problem of efficient development of task scheduling in a scenario where a large number of batch tasks are interdependent, realize hierarchical management of task scheduling by hierarchical classification of task models, realize the scheduling and scheduling of task dependencies in complex scenarios and the visualization of task relationships; drive task scheduling based on a set of general workflows, and business developers do not need to pay attention to the drawing of specific flow charts, thereby lowering the development threshold; through the granularization of batch tasks, the original time-consuming large tasks are disassembled into different small tasks and small jobs, and the multi-level task model makes the batch task execution unit more refined, which is conducive to the realization of distributed task execution and the efficient use of task scheduling resources; realize data isolation and system isolation among task scheduling, task management and batch data, which is conducive to business developers focusing on batch task processing of business; break through the scheduling rules of traditional CRON timing expressions, and formulate diversified scheduling rules according to business systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0046] Figure 1 A schematic diagram of a batch task scheduling method based on a general workflow of the present invention;

[0047] Figure 2 This is a schematic diagram of the structure of a batch task scheduling system based on a general workflow of the present invention;

[0048] Figure 3 This is a diagram of an architecture deployment example of the present invention;

[0049] Figure 4 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. It should be understood that the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments, and these embodiments are only used to illustrate the present application and are not used to limit the scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0051] like Figure 1 As shown, the embodiment of the present invention discloses a batch task scheduling method based on a general workflow, wherein the method can be implemented by applying to a scheduling platform, a task management component, and a batch processing component. The mobilization of batch tasks is driven by the scheduling platform. The scheduling platform and the task management component interact through HTTP. Different processes call different task management interfaces of the task management component to drive the task process. Specifically, the steps of the process include:

[0052] Step S1. Construct a task model, wherein the task model includes a task package, a task group, a task layer, a task, and an operation.

[0053] In the embodiment of the present invention, according to the different roles played by different levels of task scheduling, the task model is divided into five hierarchical dimensions: task package, task group, task layer, task, and job. Each level can solve different problems in the scheduling process.

[0054] Among them, the task package is mainly used to distinguish the classification management of task processes of different businesses, similar to the role of folders in a file manager. The task package for classified business management is convenient for centralized maintenance and management; the task layer mainly describes the dependency relationship between tasks. Each task layer can be configured with multiple synchronous or asynchronous tasks. Only after all synchronous tasks of the upstream task layer are completed can the downstream task layer be executed. Multiple task layers constitute the task process, which is the scheduling basis for task group scheduling; the task is configured with task attributes, and the task attributes include task type, whether, number of task slices, etc. For example, whether the task is a year-end settlement task, whether it will be postponed to the next working day after a holiday, that is, the "whether it is a year-end settlement task" attribute is configured to be true. Even if it is a holiday-delayed task, it will not be postponed if the last day of the year is a holiday. In addition, the task will also be configured with the number of task slices.

[0055] Step S2: Obtain task configuration, parse the task configuration into a task group, call the task group judgment process cyclically, judge whether the task group can be executed, and enter the execution process of the task group.

[0056] In the embodiment of the present invention, the master control calls the task configuration acquisition process, and the task configuration acquisition process can obtain the task configuration from the task management component and load it into the scheduling platform at regular intervals of 10 minutes, wherein the time interval is adjustable and configurable. Prior to this step, it also includes: the master control process of the scheduling platform has two triggering paths, the first path is triggered when the process is first started, and the second path is triggered at regular intervals every day, and the fixed time is configurable.

[0057] In this embodiment, the task configuration loaded into the scheduling platform is parsed into a task group list according to the dimension of the task group, and the task group judgment process is called through multiple task groups to determine whether the task group can be executed. The task group judgment process determines whether it can be executed on that day by calling the execution day judgment of the task group of the task management component. The judgment is mainly based on the scheduling rules configured in the task group dimension, and different scheduling rules based on natural days, working days, daily / weekly / monthly / yearly, etc. can be implemented.

[0058] Specifically, the task group judgment process is called cyclically to judge whether the task group can be executed, and the execution process of the task group is entered, including:

[0059] Calling the execution day of the task group, and judging whether the task group can be executed on that day according to the scheduling rule of the task group, wherein the scheduling rule includes a specified execution day and execution frequency, and the scheduling rule is configurable;

[0060] If the current day is not the execution day of the task group, the execution process of the current task group is terminated and the next task group is polled;

[0061] When the current day is the execution day of the task group, the execution process of the current task group is entered.

[0062] The execution process of each task group will disassemble the task sequence in the task process arranged by the task group at different task layers, and execute the task layer with high priority first. All tasks of the next task layer can only be executed after all blocked tasks in the same task layer are successfully executed. Tasks in the same task layer are executed concurrently, and the process is controlled by synchronously querying task results and blocking the execution of the next task layer.

[0063] In this embodiment, the execution process of the task group includes:

[0064] The execution process of the task group decomposes the task group into task layers, sorts the task layers according to their priorities, and executes the task layers with high priorities first;

[0065] Concurrently execute tasks at the same task layer and synchronously query task execution results;

[0066] When all tasks are executed successfully, enter the next task layer.

[0067] The synchronous query task execution results include:

[0068] Synchronous polling to query the task execution result, the polling frequency is consistent with the frequency of judging whether the task can be executed;

[0069] When the task execution result is successful, query the execution result of the next task in the same task layer; when all the tasks in the same task layer have successful execution results, execute the next task layer;

[0070] When the task execution result is failure, the execution process of the task is blocked.

[0071] In this embodiment, the task result query will continuously poll the task execution result. The polling frequency is consistent with the frequency of judging whether the task can be executed in the above steps. When the task execution result obtained by polling is a final state (success / failure), the polling is terminated. After the task execution result query is successful, the query of the next task execution result of the same layer can be performed. When the query results of all blocked tasks in the same task layer are successful, the execution of the next task layer can be performed; when the query result of the task execution is a failure, the execution of the task flow will be blocked at this task layer.

[0072] Step S3. Decompose the task group to obtain a task layer, wherein the task layer is configured with multiple tasks, determine whether the task can be executed, enter the task execution process, and execute the task of the task layer.

[0073] In the embodiment of the present invention, before each task is executed, it is necessary to first determine whether the task can be executed. In this process, the task may not be scheduled immediately due to dependency, holiday configuration, etc., which will result in a delay in the task execution. The task can only be executed when it meets the execution preconditions. If the task needs to be postponed, it is postponed according to the delay time returned by the task management component, and then the judgment step is repeated until the task execution conditions are met; if the task does not need to be postponed and can be executed immediately, the next step is performed.

[0074] Specifically, determining whether the task can be executed includes:

[0075] Determining whether the task is executable according to the task configuration;

[0076] When a task is postponed, wait for the task to be executed according to the postponement time.

[0077] Step S4: Enter the job execution process, asynchronously initiate job execution, synchronously poll and query the job execution status, and when the job query result is successful, proceed to the next job execution process.

[0078] The execution of different jobs is cyclically performed according to the job sequence in the task. First, the job execution is initiated asynchronously, and then the job execution status is polled synchronously. The next job can be executed only when the job query result is successful.

[0079] like Figure 2 As shown, the embodiment of the present invention discloses a batch task scheduling system based on a general workflow, and its structure includes:

[0080] A model building module, used to build a task model, wherein the task model includes a task package, a task group, a task layer, a task, and an operation;

[0081] A configuration parsing module is used to obtain task configuration, parse the task configuration into a task group, cyclically call the task group judgment process, judge whether the task group can be executed, and enter the execution process of the task group;

[0082] A task execution module is used to decompose the task group to obtain a task layer, wherein the task layer is configured with multiple tasks, determine whether the task can be executed, enter the task execution process, and execute the task of the task layer;

[0083] The job execution module is used to enter the job execution process, asynchronously initiate job execution, synchronously poll and query the job execution status, and proceed to the next job execution process when the job query result is successful.

[0084] By using this system, the above-mentioned operation processing method can be executed and the corresponding technical effects can be achieved.

[0085] like Figure 3 As shown, the present invention also provides a deployment example, which is deployed on the existing system architecture to implement all the steps of the method in the above embodiment. Its deployment is mainly divided into three parts, including: a scheduling platform, a task management component, and a batch processing component, wherein the task management component is a load balancer similar to ELB (elastic load balancing), and the batch processing component is a batch processing application cluster. The scheduling platform drives the execution of the general task process through the process driving engine. The batch processing component instantiates multiple nodes according to the business carrying capacity. The nodes of each batch processing application mainly include a general task component and a batch processing system. The general task component cooperates with the general task process of the scheduling platform to realize the arrangement and scheduling of complex tasks. The batch processing task scheduling system based on the general workflow described in the present invention can realize the data isolation of scheduling, tasks, and data, and reduce the intrusiveness of the scheduling system to the business system.

[0086] In the embodiment of the present invention, the data storage module on the system architecture is stored in separate libraries according to the different characteristics of each component to ensure data isolation. First, the process-driven engine scheduling data of the scheduling platform is stored in the scheduling database (Schedule DB); second, the general task component stores the task configuration and execution record related data in the task database (Task DB); finally, the business data should be separated from the platform data, and the database (Batch DB) under the batch processing component is deployed separately. The above data storage layout can minimize the impact of scheduling on the business system.

[0087] The execution of the system in the present invention is driven by the scheduling platform according to the general task process to drive the execution of each node in the process. The nodes in the general task process can interact through the general task component to realize the loading of process parameters and the driving of process branches. In this deployment example, the general task is not deployed as a separate application, but is deployed as a technical component and a batch component. The following three points are mainly considered. First, the deployment is integrated in the batch component to reduce the network interaction between the general task and the batch component; second, the general task and the batch component are integrated in one application to reduce the budget of hardware system resources; finally, the general task component and the batch component are locally integrated, and plug-in can be used to achieve plug-in and play.

[0088] General tasks and batch processing are deployed in an integrated manner, and batch processing components can be deployed in multiple nodes, which can meet the requirements of the overall architecture for processing large amounts of data. Since the instance deployment environment is carried out on the container cloud, the load balancing from scheduling to application is achieved through the ELB service, and in a virtual environment, it can also be replaced by NGINX reverse proxy or other service balancing hardware devices. In the technical implementation, the HTTP communication protocol is used to implement the interaction between general task processes and general task components. The HTTP protocol is widely used across platforms, so that the selection range of the scheduling platform is larger. In the embodiment of the present invention, the scheduling platform can use the Entegor scheduling platform, and similar workflow-based scheduling platforms can also use PowerJob, XxlJob, etc.

[0089] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all the steps in the method in the above embodiment, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps in the method in the above embodiment are implemented.

[0090] The embodiment of the present invention further provides an electronic device for executing the above method, as a device for implementing the method, and its structure is as follows: Figure 4As shown, the electronic device 400 includes: a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, such as being connected via a bus 402. Optionally, the electronic device 400 may further include a transceiver 404. It should be noted that in actual applications, the transceiver 404 is not limited to one. It is understandable that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the specific structure of the electronic device 400. In other embodiments of the present application, the electronic device 400 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. Optionally, the electronic device may further include a display screen 405 for displaying images, or receiving user operation instructions when necessary.

[0091] The processor 401 is applied in the embodiment of the present application to implement the method shown in the above method embodiment. The transceiver 404 may include a receiver and a transmitter. The transceiver 404 is applied in the embodiment of the present application to implement the function of the electronic device of the embodiment of the present application communicating with other devices when executed.

[0092] Processor 401 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0093] The processor 401 may also include one or more processing units, for example, the processor 401 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the electronic device 400. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be set in the processor 401 for storing instructions and data. In some embodiments, the memory in the processor 401 is a high-speed cache memory. The memory may store instructions or data that the processor 401 has just used or circulated. If the processor 401 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 401 is reduced, and the efficiency of the system is improved.

[0094] The processor 401 can run the method provided in the embodiment of the present application. The processor 401 can include different devices. For example, when a CPU and a GPU are integrated, the CPU and the GPU can cooperate to execute the method provided in the embodiment of the present application, for example, part of the algorithm in the method is executed by the CPU, and another part of the algorithm is executed by the GPU to obtain faster processing efficiency.

[0095] The bus 402 may include a path to transmit information between the above components. The bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0096] The memory 403 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or may include a high-speed random access memory, or may include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0097] Optionally, the memory 403 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 401. The processor 401 is used to execute the application code stored in the memory 403 to implement the method provided in any embodiment of the present application.

[0098] The memory 403 can be used to store computer executable program codes, which include instructions. The processor 401 executes various functional applications and data processing of the electronic device 400 by running the instructions stored in the memory 403. The memory 403 may include a program storage area and a data storage area. Among them, the program storage area may store the code of the operating system, application program, etc. The data storage area may store data created during the use of the electronic device 400 (such as images, videos, etc. collected by the camera application), etc.

[0099] The memory 403 may also store one or more computer programs corresponding to the method provided in the embodiment of the present application. The one or more computer programs are stored in the above-mentioned memory 403 and are configured to be executed by the one or more processors 401, and the one or more computer programs include instructions, and the above-mentioned instructions can be used to execute each step in the above-mentioned corresponding embodiment.

[0100] Of course, the code of the method provided in the embodiment of the present application can also be stored in an external memory. In this case, the processor 401 can run the code of the method stored in the external memory through the external memory interface, and the processor 401 can control the running process.

[0101] The display screen 405 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 400 may include 1 or N display screens 405, where N is a positive integer greater than 1. The display screen 405 may be used to display information input by a user or information provided to a user and various graphical user interfaces (GUI). For example, the display screen 405 may display photos, videos, web pages, or files, etc.

[0102] The electronic device provided in the embodiment of the present application is applicable to any embodiment of the above method. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, which will not be repeated here.

[0103] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0105] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in a box or multiple boxes. Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0107] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A batch task scheduling method based on general workflow, characterized in that: include: S1. Constructing a task model, wherein the task model includes a task package, a task group, a task layer, a task, and a job; S2. Get the task configuration, parse the task configuration into a task group, call the task group judgment process in a loop, determine whether the task group can be executed, and enter the execution process of the task group; S3. Decomposing the task group to obtain a task layer, the task layer is configured with multiple tasks, determining whether the task can be executed, entering the task execution process, and executing the task layer tasks; S4. Enter the job execution process, asynchronously initiate job execution, synchronously poll and query the job execution status, and when the job query result is successful, proceed to the next job execution process.

2. The method according to claim 1, characterized in that The task package is used to distinguish the classification management of task processes of different businesses, the task layer is used to describe the dependency relationship between tasks, and the task is used to configure task attributes, which include task type, whether to postpone, and the number of task slices.

3. The method according to claim 1, characterized in that The cyclically calling task group judgment process, judging whether the task group can be executed, and entering the execution process of the task group, includes: Calling the execution day of the task group, and judging whether the task group can be executed on that day according to the scheduling rule of the task group, wherein the scheduling rule includes a specified execution day and execution frequency, and the scheduling rule is configurable; If the current day is not the execution day of the task group, the execution process of the current task group is terminated and the next task group is polled; When the current day is the execution day of the task group, the execution process of the current task group is entered.

4. The method according to claim 1, characterized in that The execution process of the task group includes: The execution process of the task group decomposes the task group into task layers, sorts the task layers according to their priorities, and executes the task layers with high priorities first; Concurrently execute tasks at the same task layer and synchronously query task execution results; When all tasks are executed successfully, enter the next task layer.

5. The method according to claim 4, characterized in that The synchronous query task execution result includes: Synchronous polling to query the task execution result, the polling frequency is consistent with the frequency of judging whether the task can be executed; When the task execution result is successful, query the execution result of the next task in the same task layer; when all the tasks in the same task layer have successful execution results, execute the next task layer; When the task execution result is failure, the execution process of the task is blocked.

6. The method according to claim 1, characterized in that The determining whether the task can be executed includes: Determining whether the task is executable according to the task configuration; When a task is postponed, wait for the task to be executed according to the postponement time.

7. A batch task scheduling system based on general workflow, characterized in that: include: A model building module, used to build a task model, wherein the task model includes a task package, a task group, a task layer, a task, and an operation; A configuration parsing module is used to obtain task configuration, parse the task configuration into a task group, cyclically call the task group judgment process, judge whether the task group can be executed, and enter the execution process of the task group; A task execution module is used to decompose the task group to obtain a task layer, wherein the task layer is configured with multiple tasks, determine whether the task can be executed, enter the task execution process, and execute the task of the task layer; The job execution module is used to enter the job execution process, asynchronously initiate job execution, synchronously poll and query the job execution status, and proceed to the next job execution process when the job query result is successful.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store program instructions; The processor is configured to execute the method according to any one of claims 1 to 6 by calling program instructions.

10. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images