A unified scheduling method and system for multiple workflow engines

Abstract

This invention discloses a unified management and scheduling method, system, and storage medium for multiple workflow engines, belonging to the field of workflow orchestration and task scheduling technology. The method includes: receiving a start request for a target workflow, creating and recording a workflow instance, and determining the engine type; based on the engine type, selecting a target runtime adapter, driving the workflow instance to execute in the engine, and establishing a node execution record; when a target workflow node enters a waiting state, writing waiting and recovery information into an extended storage area; upon receiving a recovery advancement request, resuming the template workflow's progress, and verifying and auditing the recovery data; submitting the verified recovery data to the target runtime adapter, driving the target workflow node to complete the recovery advancement and triggering subsequent node scheduling, and outputting unified progress data. This invention can reduce the integration cost when multiple engines coexist, and improve cross-engine collaborative scheduling capabilities and audit traceability capabilities.

Description

Technical Field

[0001] This invention relates to a unified scheduling method and system for multiple workflow engines, belonging to the field of process orchestration and task scheduling technology. Background Technology

[0002] In enterprise-level software systems, process orchestration and task scheduling are used to automate complex business processes across modules and teams. In engineering practice, due to factors such as ecosystem maturity, scalability, usage habits, or historical reasons, different teams may simultaneously introduce multiple open-source scheduling tools or workflow engines and overlay a task layer within the business system to carry business semantics and data processing logic.

[0003] Due to the differences in conceptual models, state machines, pause and wait methods, recovery and progress methods, and log tracing mechanisms among different engines, the following problems are likely to occur when multiple engines coexist: (1) The initiation entry, operation monitoring, pause and recovery and tracing methods of the same business are different under different engines, making it difficult for the platform to manage them uniformly; (2) Business systems often need to repeatedly implement state mapping, operation docking, wait and recovery closed loop and execution record storage for different engines, which significantly increases integration costs and operation and maintenance risks; (3) The implementation of links such as manual review and sub-process collaboration that "require external intervention" varies greatly in different engines, making it difficult to form a consistent and auditable wait-recovery semantics, which affects user experience and governance capabilities.

[0004] Therefore, it is necessary to propose an engine-agnostic unified management and scheduling scheme, so that various open-source scheduling tools can achieve consistent operation governance, closed-loop interaction and traceable records under a unified scheduling layer. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a unified scheduling method and system for multiple workflow engines. This method and system can converge key capabilities such as process instances, node records, waiting recovery, distribution routing and progress query to a unified scheduling layer without limiting specific business or forcibly replacing existing open source scheduling tools. In evaluation-type processes, "creating task nodes" is used as a unified startup entry point, thereby reducing the integration cost when multiple engines coexist and improving cross-engine collaborative scheduling capabilities and audit traceability capabilities.

[0006] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, this invention discloses a unified scheduling method for multiple workflow engines, comprising: Receive the start request of the target process, create a process instance, and determine the engine type of the workflow engine of the process instance; the workflow engine is the underlying scheduling engine that the process instance actually executes. Based on the engine type, a target runtime adapter is determined to drive the process instance to execute in the corresponding workflow engine and output unified progress data. The unified progress data includes the node status of the node execution record and the interactive information in the extended storage area for external display. During execution, node execution records are established for multiple process nodes in the process instance. The node execution record includes node status, node output, and an extended storage area for storing interactive information and recovery entry description information. During the execution process, when the target process node in the process instance enters a waiting state, the waiting and recovery information of the target process node is stored in the extended storage area. Upon receiving a recovery request, the target process instance and target process node are determined based on the callback description information carried in the recovery request. The recovery route is determined based on the target process instance, target process node, and the stored waiting and recovery information of the target process node. Recovery data is extracted based on the recovery request of the recovery route, and the recovery data is verified and audited. The verified recovery data is submitted to the target runtime adapter to drive the target process node to complete the recovery process and trigger the scheduling of subsequent process nodes.

[0007] Furthermore, the target process node enters a waiting state, including, When the target process node meets the set external interaction conditions, the target process node is set to a waiting state; the set external interaction conditions include the target process node needing manual review or needing to wait for the sub-process to complete.

[0008] Furthermore, the callback description information includes a trigger source type identifier and a process instance identifier and a process node identifier used to locate the waiting process node; the trigger source type identifier includes manual review result feedback and sub-process completion feedback.

[0009] Furthermore, the runtime adapter includes multiple adapters that interface with open-source scheduling engines, and interacts with these multiple adapters using a unified interface.

[0010] Furthermore, the verification of the recovered data includes, Based on the pre-declared input parameter structure and field constraint model of the target process node, the structure and field constraint of the restored data are validated; if the validation fails, the restoration process is rejected and an error message is returned.

[0011] Furthermore, the waiting state and the recovery process satisfy idempotent control, including: When the target process node is not in a waiting state, the resumption process is refused and the refusal is recorded in the audit log.

[0012] Furthermore, the extended storage area uses namespaces to organize interactive information to isolate interactive contexts for different purposes; when outputting unified progress data externally, the internal fields in the extended storage area are cleaned up and hidden.

[0013] Furthermore, the execution process also includes cross-engine parent-child process collaboration, specifically: When a parent process instance triggers a child process instance, the child process instance records the parent process instance identifier and the parent process node identifier, and the parent process node enters a waiting state. When a child process instance completes, a callback description is generated to restore the parent process node, and the restoration of the parent process node is triggered, thus completing the collaborative scheduling of the parent and child processes across different workflow engines.

[0014] Furthermore, the waiting and recovery information includes a waiting interaction context and recovery location information; wherein, the waiting interaction context includes the waiting reason and external display information, and the recovery location information includes the process instance identifier and the process node identifier; The interactive information includes the reason for waiting, the description of the task to be completed, the optional action items, or the result summary for external display.

[0015] Secondly, this invention discloses a unified scheduling system for multiple workflow engines, comprising: The receiving module is used to receive the start request of the target process, create a process instance, and determine the engine type of the workflow engine of the process instance; the workflow engine is the underlying scheduling engine that the process instance actually executes. An execution module is used to determine a target runtime adapter based on the engine type, so as to drive the process instance to execute in the corresponding workflow engine and output unified progress data. The unified progress data includes the node status of the node execution record and the interactive information in the extended storage area for external display. During the execution process, node execution records are established for multiple process nodes in the process instance. The node execution record includes node status, node output, and an extended storage area for storing interactive information and recovery entry description information. The waiting and recovery module is used to, during the execution process, when a target process node in the process instance enters a waiting state, store the waiting and recovery information of the target process node in the extended storage area; after receiving a recovery advancement request, determine the target process instance and target process node according to the callback description information carried in the recovery advancement request, determine the recovery route according to the target process instance, target process node and the stored waiting and recovery information of the target process node, extract recovery data based on the recovery request of the recovery route, and verify and audit the recovery data; submit the verified recovery data to the target runtime adapter to drive the target process node to complete the recovery advancement and trigger the scheduling of subsequent process nodes.

[0016] The beneficial effects achieved by this invention are as follows: The method provided by this invention, through a unified scheduling layer and runtime adaptation mechanism, can uniformly manage multiple open-source scheduling tools, enabling business systems to complete startup, monitoring, and governance under a unified entry point; through a unified wait-recovery semantics and distribution routing mechanism, it can form a consistent closed loop for external intervention links such as manual review and sub-process collaboration, reducing redundant development for different engines; through a unified node execution record and extended storage area, it can achieve structured retention of execution context and audit information, enhancing traceability and compliance governance capabilities; and through a cross-engine parent-child process collaboration mechanism, it can improve the organizational efficiency and scalability of complex evaluation processes in engineering practice. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a unified scheduling method for multiple workflow engines provided by the present invention.

[0018] Figure 2 This invention provides an architecture diagram of a unified scheduling system for multiple workflow engines. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0020] Example 1, as Figure 1 As shown, this invention discloses a unified scheduling method for multiple workflow engines, comprising: Receive the start request of the target process, create a process instance, and determine the engine type of the workflow engine of the process instance; the workflow engine is the underlying scheduling engine that the process instance actually executes. Based on the engine type, a target runtime adapter is determined to drive the process instance to execute in the corresponding workflow engine and output unified progress data. The unified progress data includes the node status of the node execution record and the interactive information in the extended storage area for external display. During execution, node execution records are established for multiple process nodes in the process instance. The node execution record includes node status, node output, and an extended storage area for storing interactive information and recovery entry description information. During the execution process, when the target process node in the process instance enters a waiting state, the waiting and recovery information of the target process node is stored in the extended storage area. Upon receiving a recovery request, the target process instance and target process node are determined based on the callback description information carried in the recovery request. The recovery route is determined based on the target process instance, target process node, and the stored waiting and recovery information of the target process node. Recovery data is extracted based on the recovery request of the recovery route, and the recovery data is verified and audited. The verified recovery data is submitted to the target runtime adapter to drive the target process node to complete the recovery process and trigger the scheduling of subsequent process nodes.

[0021] The target process node entering the waiting state includes, When the target process node meets the set external interaction conditions, the target process node is set to a waiting state; The pre-set external interaction conditions include whether the target process node requires manual review or needs to wait for the sub-process to complete.

[0022] The recovery process includes, The target process instance and target node are determined based on the callback description information carried in the recovery request, and the recovery route is determined accordingly. The callback description information includes a trigger source type identifier and a process instance identifier and node identifier used to locate the waiting node; The trigger source type identifier includes manual review result feedback and sub-process completion feedback.

[0023] The verification of the recovered data includes, Based on the pre-declared input parameter structure and field constraint model of the target process node, the structure and field constraint of the recovered data are verified. If the verification fails, the process will not be resumed and an error message will be returned.

[0024] The waiting state and the recovery process are subject to idempotent control. When the target process node is not in a waiting state, the recovery process is refused to be executed on the node, and the refusal is recorded in the audit log.

[0025] The extended storage area organizes interactive information using namespaces to isolate interactive contexts for different purposes; when outputting unified progress data externally, the internal fields in the extended storage area are cleaned up and hidden.

[0026] Example 2 introduces a unified scheduling method for multiple workflow engines. Taking the evaluation process as an example, this example provides the method steps for the wait-recovery closed loop of the present invention. These steps can be completed collaboratively by the unified scheduling layer, runtime adapter, and data storage, including: S1. When the "create task node" action is generated within the business system, this action is used as a process start trigger signal. The unified scheduling layer receives the trigger signal, creates a process instance, records the engine type, and writes the basic information of the process instance.

[0027] S2. The unified scheduling layer selects the corresponding runtime adapter based on the recorded engine type, calls the adapter to trigger the creation and operation of the target open-source scheduling engine, and writes the engine-side running identifier and startup result back to the process instance.

[0028] S3. The unified scheduling layer establishes node execution records based on the node structure of the process, initializes the node state, and sets up an extended storage area for the node execution records.

[0029] S4. When the process reaches the target node and meets the external interaction conditions such as manual review or waiting for the sub-process to complete, the target node is set to a waiting state, and the waiting interaction context and recovery entry description information are written to the extended storage area. The waiting interaction context includes at least the waiting reason, the to-do description, and the external display information, and the recovery entry description information includes at least the callback description information used to locate the recovery object.

[0030] S5. The unified scheduling layer receives a recovery advancement request triggered internally by the system; the source of the recovery advancement request includes at least: the audit result generated by the audit center, or the sub-process completion event; the unified scheduling layer reads the callback description information carried in the request, determines the recovery route and target waiting node, and writes it into the audit record.

[0031] S6. The unified scheduling layer verifies the recovery data. The verification includes at least structural and field constraint verification based on the input data model declared by the target node. When the verification fails, the recovery process is rejected and an error message is returned. At the same time, the failure audit is recorded.

[0032] S7. After the verification is passed, the unified scheduling layer performs idempotent control: recovery is allowed only when the target node is in a waiting state; then the recovery data is submitted to the corresponding runtime adapter, which drives the target open source scheduling engine to complete the recovery and update the node output and node status.

[0033] S8. After the recovery process is completed, the adapter triggers the subsequent node to continue scheduling; the unified scheduling layer continuously updates the process instance and node execution record, and outputs unified progress data to the outside world. The unified progress data includes at least the node status and interactive information that can be displayed to the outside world.

[0034] S9. When a parent process triggers a child process, the parent process identifier and parent node identifier are recorded in the child process instance, and the hierarchical relationship is also recorded; the child process can choose to execute with a different engine type than the parent process.

[0035] S10. The parent node enters a waiting state and writes callback description information for restoring the parent node in the extended storage area.

[0036] S11. After the sub-process is completed, the system generates a recovery advance request, carrying callback description information for locating the parent node and a summary of the sub-process output.

[0037] S12. The unified scheduling layer selects the corresponding adapter to restore the parent node based on the parent process engine type and drives the parent process to continue execution, thereby realizing cross-engine combined scheduling.

[0038] Among them, such as Figure 2 As shown, an example of the unified scheduling layer maintenance process and node execution records are presented: A process instance is used to record basic information about an end-to-end process run, including at least the selected workflow engine type (open source scheduling engine A, open source scheduling engine B), and can be associated with an external engine run identifier; The node execution log is used to record node-level status and output, and provides an extended storage area for storing interaction context, recovery entry description information, and audit information.

[0039] The extended storage area is used to hold critical clues needed for recovery, such as the reason for waiting, information to be displayed externally, recovery entry point location information (process instance identifier, node identifier, trigger source type identifier), and verification basis identifier. To reduce data overlap between different purposes, in some implementations, the extended storage area is organized using namespaces, which are used for manual review waiting, sub-process waiting, and audit clues, respectively.

[0040] When the unified scheduling layer outputs the process progress to the outside world, it can clean up and hide the internal fields in the extended storage area, retaining only the information related to user interaction. This allows the front end to read the necessary waiting clues and interaction context without exposing the internal operating details.

[0041] The unified scheduling layer maintains a set of runtime adapters, with different adapters connecting to different open-source scheduling engines. When a process is started, its progress is queried, or its progress is resumed, the unified scheduling layer selects the appropriate adapter to execute based on the engine type recorded in the process instance.

[0042] Using this approach, the business side does not need to understand the differences in the underlying engines. The unified scheduling layer provides consistent process initiation, progress query, and recovery advancement capabilities. For different open-source engines, this invention does not require changes to their original scheduling logic. Instead, the adapter completes the necessary information conversion and call coordination at the boundary, such as triggering execution, status query, and recovery advancement of waiting nodes.

[0043] The implementation methods for two types of waiting scenarios (manual review and sub-process waiting) are as follows: The manual review waiting process works as follows: When the process reaches a node requiring manual judgment, the unified scheduling layer sets the node to wait and writes the necessary interactive information for review into the extended storage area, such as review instructions, a summary of the pending review results, and optional conclusion items. After the reviewer completes the operation in the review center, the review center triggers a recovery request within the system, carrying the review conclusion and necessary data. After the unified scheduling layer completes verification, auditing, and idempotency control, it advances the node to continue execution, thereby enabling the structured writing of the manual decision result into the execution record and driving subsequent nodes.

[0044] The sub-process waiting process is as follows: When the parent process reaches the "sub-process trigger node," the unified scheduling layer creates a sub-process instance and puts the parent node into a waiting state. Simultaneously, callback description information for subsequent recovery is written to the extended storage area. After the sub-process completes its execution, its completion event is captured internally and triggers a recovery advancement request. Based on this, the unified scheduling layer restores the parent node and continues scheduling subsequent nodes of the parent process, thereby achieving process decomposition and coordination.

[0045] Callback description information and distribution routes include: adapting to recovery triggers from different sources within the system. This invention abstracts the information required for the recovery entry point into callback description information. The callback description information includes at least a trigger source type identifier, a process instance identifier, and a node identifier, used to distinguish the recovery trigger source and locate the target waiting node.

[0046] The unified scheduling layer distributes data based on callback description information: when the callback description information points to a waiting node, the unified scheduling layer reads the engine type from the process instance and selects the corresponding adapter to execute the recovery process; when the recovery is triggered by a subprocess completion event, the unified scheduling layer can use the subprocess output summary as part of the recovery input to drive the parent node to calculate the output and continue execution.

[0047] By adopting this mechanism, different modules within the system (such as the audit center and sub-process event handling modules) can trigger recovery in a consistent manner, avoiding the need to implement separate recovery waiting logic for different engines.

[0048] In practical implementation, the unified scheduling layer is integrated with the platform's authentication system. Recovery requests carry operator or system identity information, and the unified scheduling layer audits and records the recovery operation, including recovery time, operator identification, and recovery result, for subsequent traceability. It should be noted that the specific implementation of auditing and permissions can be consistent with the platform's existing system; this invention focuses on maintaining a traceable record link within the multi-engine management and recovery waiting closed loop.

[0049] Example 3: This example introduces a unified scheduling system for multiple workflow engines, including: The receiving module is used to receive the start request of the target process, create a process instance, and determine the engine type of the workflow engine of the process instance; the workflow engine is the underlying scheduling engine that the process instance actually executes. An execution module is used to determine a target runtime adapter based on the engine type, so as to drive the process instance to execute in the corresponding workflow engine and output unified progress data. The unified progress data includes the node status of the node execution record and the interactive information in the extended storage area for external display. During the execution process, node execution records are established for multiple process nodes in the process instance. The node execution record includes node status, node output, and an extended storage area for storing interactive information and recovery entry description information. The waiting and recovery module is used to, during the execution process, when a target process node in the process instance enters a waiting state, store the waiting and recovery information of the target process node in the extended storage area; after receiving a recovery advancement request, determine the target process instance and target process node according to the callback description information carried in the recovery advancement request, determine the recovery route according to the target process instance, target process node and the stored waiting and recovery information of the target process node, extract recovery data based on the recovery request of the recovery route, and verify and audit the recovery data; submit the verified recovery data to the target runtime adapter to drive the target process node to complete the recovery advancement and trigger the scheduling of subsequent process nodes.

[0050] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0051] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0052] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0053] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0054] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A unified scheduling method for multiple workflow engines, characterized in that, include: Receive the start request of the target process, create a process instance, and determine the engine type of the workflow engine for the process instance; The workflow engine is the underlying scheduling engine that actually executes the process instance; Based on the engine type, a target runtime adapter is determined to drive the process instance to execute in the corresponding workflow engine and output unified progress data; The unified progress data includes the node status of the node execution record and the interactive information in the extended storage area used for external display; During execution, node execution records are established for multiple process nodes in the process instance; the node execution records include node status, node output, and an extended storage area for storing interaction information and recovery entry description information. During the execution process, when the target process node in the process instance enters a waiting state, the waiting and recovery information of the target process node is stored in the extended storage area. Upon receiving a recovery request, the target process instance and target process node are determined based on the callback description information carried in the recovery request. The recovery route is determined based on the target process instance, target process node, and the stored waiting and recovery information of the target process node. Recovery data is extracted based on the recovery request of the recovery route, and the recovery data is verified and audited. The verified recovery data is submitted to the target runtime adapter to drive the target process node to complete the recovery process and trigger the scheduling of subsequent process nodes.

2. The unified scheduling method for multiple workflow engines according to claim 1, characterized in that, The target process node enters a waiting state, including: When the target process node meets the set external interaction conditions, the target process node is set to a waiting state; the set external interaction conditions include the target process node needing manual review or needing to wait for the sub-process to complete.

3. The unified scheduling method for multiple workflow engines according to claim 1, characterized in that, The callback description information includes a trigger source type identifier and a process instance identifier and a process node identifier used to locate waiting process nodes; the trigger source type identifier includes manual review result feedback and sub-process completion feedback.

4. The unified scheduling method for multiple workflow engines according to claim 1, characterized in that, The runtime adapter includes various adapters that interface with open-source scheduling engines, and interacts with these adapters using a unified interface.

5. The unified scheduling method for multiple workflow engines according to claim 1, characterized in that, The verification of the recovered data includes, Based on the pre-declared input parameter structure and field constraint model of the target process node, the structure and field constraint of the recovered data are verified. If the verification fails, the process will not be resumed and an error message will be returned.

6. The unified scheduling method for multiple workflow engines according to claim 1, characterized in that, The waiting state and the recovery process satisfy idempotent control, including: When the target process node is not in a waiting state, the resumption process is refused and the refusal is recorded in the audit log.

7. The unified scheduling method for multiple workflow engines according to claim 1, characterized in that, The extended storage area organizes interactive information using namespaces to isolate interactive contexts for different purposes; when outputting unified progress data externally, the internal fields in the extended storage area are cleaned up and hidden.

8. The unified scheduling method for multiple workflow engines according to claim 1, characterized in that, The execution process also includes cross-engine parent-child process collaboration, specifically: When a parent process instance triggers a child process instance, the child process instance records the parent process instance identifier and the parent process node identifier, and the parent process node enters a waiting state. When a child process instance completes, a callback description is generated to restore the parent process node, and the restoration of the parent process node is triggered, thus completing the collaborative scheduling of the parent and child processes across different workflow engines.

9. The unified scheduling method for multiple workflow engines according to claim 1, characterized in that, The waiting and recovery information includes waiting interaction context and recovery location information; wherein, the waiting interaction context includes the reason for waiting and the information displayed externally, and the recovery location information includes the process instance identifier and the process node identifier; The interactive information includes the reason for waiting, the description of the task to be completed, the optional action items, or the result summary for external display.

10. A unified scheduling system for multiple workflow engines, characterized in that, include: The receiving module is used to receive the start request of the target process, create a process instance, and determine the engine type of the workflow engine of the process instance. The workflow engine is the underlying scheduling engine that actually executes the process instance; The execution module is used to determine the target runtime adapter based on the engine type, so as to drive the process instance to execute in the corresponding workflow engine and output unified progress data; The unified progress data includes the node status of the node execution record and the interactive information in the extended storage area used for external display; During execution, node execution records are established for multiple process nodes in the process instance; the node execution records include node status, node output, and an extended storage area for storing interaction information and recovery entry description information. The waiting and recovery module is used to, during the execution process, when a target process node in the process instance enters a waiting state, store the waiting and recovery information of the target process node in the extended storage area; after receiving a recovery advance request, determine the target process instance and target process node according to the callback description information carried by the recovery advance request, determine the recovery route according to the target process instance, target process node and the stored waiting and recovery information of the target process node, extract recovery data based on the recovery request of the recovery route, and perform verification and auditing records on the recovery data; The verified recovery data is submitted to the target runtime adapter to drive the target process node to complete the recovery process and trigger the scheduling of subsequent process nodes.

Images