Distributed workflow scheduling system and method

Abstract

The application provides a distributed workflow scheduling system and method, which is applied to the technical field of distribution and comprises at least one working node, a registration node and a control node, wherein the working node is used for sending a service interface registration instruction to the registration node, submitting a workflow task to the control node, receiving a subtask distributed by the control node and executing; the registration node is used for registering the service interface of the working node after receiving the service interface registration instruction, and sending the service interface information of the registered working node to the control node; the control node is used for receiving the service interface information of the working node, receiving the workflow task of the working node, determining the working node for executing each subtask, sending the subtask to the working node through the service interface of the working node, and feeding back through the service interface of the working node corresponding to the workflow task after obtaining the execution result of the workflow task. The application can realize distributed workflow scheduling and improve the processing speed of complex business.

Description

Technical Field

[0001] This invention relates to the field of distributed technology, and in particular to a distributed workflow scheduling system and method. Background Technology

[0002] In recent years, with the continuous development of banking and financial services, the requirements for information technology construction have become increasingly stringent. Workflow technology is being applied more and more widely in banking business systems. Many banking financial operations are characterized by standardization, process-orientation, and controllability, and workflow technology can effectively adapt to and solve these problems. On the other hand, based on the increasingly widespread use of business systems and the increasing transaction volume, banking systems are gradually transforming into distributed architectures. Distributed architectures have their own complexity and efficiency characteristics. In order to quickly respond to customer requests, solve the problems of high concurrency in business systems, and improve the operational efficiency of business systems, combining workflow technology and distributed technology is an urgent problem to be solved. However, there is currently no effective solution for combining the two. Summary of the Invention

[0003] This invention proposes a distributed workflow scheduling system to combine distributed systems and workflows, thereby achieving distributed workflow scheduling, improving the processing speed of complex business processes, and enhancing system operating efficiency. The system includes:

[0004] At least one worker node is used to send service interface registration instructions to the registration node, submit workflow tasks to the management node, receive sub-tasks assigned by the management node and execute them.

[0005] The registration node is used to register the service interface of the worker node after receiving the service interface registration instruction from the worker node, and to send the registered service interface information of the worker node to the management node.

[0006] The control node is used to receive service interface information from the worker nodes; receive workflow tasks from at least one worker node; determine the worker node to execute each subtask based on the execution order and execution status of the subtasks in all workflow tasks; and send the subtask to the worker node through its service interface; and provide feedback through the service interface of the worker node corresponding to the workflow task after obtaining the execution result of the workflow task.

[0007] This invention proposes a distributed workflow scheduling method to implement distributed workflow scheduling. The method includes:

[0008] Receive the service interface information of the worker node. The service interface information of the worker node is sent by the registry center after registering the service interface of the worker node after receiving the service interface registration instruction of the worker node.

[0009] Receive workflow tasks from at least one worker node;

[0010] Based on the execution order and execution status of the subtasks in all workflow tasks, determine the work node for executing each subtask;

[0011] The subtask is sent to the worker node through the worker node's service interface, and the worker node executes the subtask upon receiving it.

[0012] After obtaining the execution result of the workflow task, feedback is provided through the service interface of the corresponding work node of the workflow task.

[0013] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described distributed workflow scheduling method.

[0014] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described distributed workflow scheduling method.

[0015] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described distributed workflow scheduling method.

[0016] In this embodiment of the invention, at least one worker node is used to send a service interface registration instruction to a registration node, submit workflow tasks to a management node, and receive and execute sub-tasks assigned by the management node. The registration node, upon receiving the service interface registration instruction from a worker node, registers the worker node's service interface and sends the registered worker node's service interface information to the management node. The management node receives the worker node's service interface information; receives workflow tasks from at least one worker node; determines the worker node to execute each sub-task based on the execution order and execution status of all sub-tasks in the workflow tasks; and sends the sub-task to the worker node through its service interface. After obtaining the execution result of the workflow task, feedback is provided through the service interface of the corresponding worker node. In this process, by splitting and scheduling workflow tasks from multiple worker nodes through the management node and managing the service interfaces of worker nodes through the registration center, the processing speed of complex business processes can be improved, and system operating efficiency can be enhanced. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0018] Figure 1 This is a schematic diagram of a distributed workflow scheduling system in an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the Dubbo-zookeeper framework in an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of the control node in an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram illustrating the execution order of subtasks in an embodiment of the present invention;

[0022] Figure 5 This is a flowchart of the distributed workflow scheduling method in an embodiment of the present invention;

[0023] Figure 6 This is a schematic diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0025] In the description of this specification, the terms "comprising," "including," "having," and "containing" are open-ended terms, meaning that they include but are not limited to. The terms "an embodiment," "a specific embodiment," "some embodiments," and "for example," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. The order of steps involved in the various embodiments is used to illustrate the implementation of this application, and the order of steps is not limited and can be adjusted appropriately as needed.

[0026] Figure 1 This is a schematic diagram of a distributed workflow scheduling system in an embodiment of the present invention, such as... Figure 1 As shown, it includes:

[0027] At least one worker node (workernode101) is used to send service interface registration instructions to the registration node, submit workflow tasks to the management node, receive sub-tasks assigned by the management node, and execute them.

[0028] Registration node 102 is used to register the service interface of the worker node after receiving the service interface registration instruction of the worker node, and send the registered service interface information of the worker node to the management node.

[0029] The control node 103 is used to receive service interface information from the worker nodes; receive workflow tasks from at least one worker node; determine the worker node to execute each subtask based on the execution order and execution status of the subtasks in all workflow tasks; and send the subtask to the worker node through the service interface of the worker node; and provide feedback through the service interface of the worker node corresponding to the workflow task after obtaining the execution result of the workflow task.

[0030] In the above embodiments, the workflow task is created by the worker node after receiving a user request.

[0031] In one embodiment, the control node integrates a workflow engine.

[0032] Workflow refers to the automated execution of part or all of a business process in a computer application environment. It is an abstract and generalized description of a workflow and the business rules governing its various operational steps. To achieve a specific business objective, a workflow system utilizes computers to automatically transfer context, information, or task parameters among multiple participants according to predetermined rules.

[0033] There are various workflow engines available today, such as the popular jBPM, Activiti, and Snaker. Activiti is a commonly used engine, characterized by its ultra-fast and stable BPMN2 workflow engine, which is easy to integrate with Spring. Activiti provides a web interface for workflow design and generates corresponding configuration files, making it easy to develop and maintain. This invention uses the Activiti workflow engine for design and development.

[0034] In one embodiment, the registry center is ZooKeeper. Currently, there are many distributed frameworks, including Spring Cloud and Dubbo-ZooKeeper, each with its own advantages and disadvantages. The basic components of a distributed architecture include remote call brokers, ZooKeeper registry centers, rate limiters, load balancers, etc., which publish and register interfaces with the registry center to achieve features such as remote calls.

[0035] Figure 2 This is a schematic diagram of the Dubbo-zookeeper framework in an embodiment of the present invention. The provider publishes its own interfaces to the registry center, and the consumer obtains the corresponding interface list through the registry center and makes remote calls. Simultaneously, Dubbo provides a monitoring interface to collect statistics on interface call counts, execution times, etc.

[0036] Figure 3 This is a schematic diagram of a control node in an embodiment of the present invention. In one embodiment, the control node includes:

[0037] The management module 301 is used to break down workflow tasks into subtasks and manage the execution status and progress of all subtasks.

[0038] The decision module 302 is used to determine the work node for executing each subtask based on the execution order and execution status of the subtasks in all workflow tasks;

[0039] The communication module 303 is used to communicate with the registered node and the worker node, receive service interface information from the worker node; receive workflow tasks from at least one worker node; send subtasks to the worker node through the worker node's service interface; and provide feedback on the execution result of the workflow task through the service interface of the corresponding worker node.

[0040] The execution module 304 is used to send the subtask to be executed to the corresponding working node through the communication module according to the execution order and execution status of the subtask; upon receiving the execution result from the corresponding working node, it records the execution status of the subtask and determines the next subtask to be executed.

[0041] Figure 4 This is a schematic diagram illustrating the execution order of subtasks in an embodiment of the present invention. Some subtasks can be executed simultaneously in parallel, while others need to wait for the previous subtask or a group of subtasks to complete before they can be executed.

[0042] In the above embodiments, the management module 301 can manage the execution status and progress of all subtasks. Therefore, the decision module can design various strategies to distribute subtasks and ensure that each subtask is executed efficiently and in an orderly manner.

[0043] In one embodiment, the working node is further configured to:

[0044] System information is periodically sent to the control node, including one or any combination of CPU utilization, network connection count, and memory utilization.

[0045] See Figure 2The control node also includes a system information collection module 305, which is used to collect system information of the registered node through the communication module;

[0046] The decision module is also used to determine the working node for executing each subtask based on the execution order and status of subtasks in all workflow tasks, as well as the system information of the registered nodes.

[0047] In one embodiment, see Figure 2 The control node also includes a storage module 306, which is used to store system information, service interface information of worker nodes, execution order and execution status of subtasks.

[0048] In one embodiment, see Figure 2 The control node also includes a web interface module 307, which is used to display the execution status and progress of workflow tasks.

[0049] In one embodiment, the web interface module is further configured to: receive a user's click operation instruction, the click operation instruction including a subtask cancellation instruction, a subtask trigger instruction, or a subtask re-execution instruction;

[0050] The communication module is also used to: send a subtask stop command to the worker node executing the subtask when the click operation command is a subtask cancellation command; send a subtask start command to the worker node executing the subtask when the click operation command is a subtask trigger command; and send a subtask restart command to the worker node executing the subtask when the click operation command is a subtask re-execution command.

[0051] In one embodiment, see Figure 2 The control node also includes a work node status management module 308, which is used to monitor the status of work nodes through a heartbeat mechanism and generate reminder information when the status of a work node is abnormal.

[0052] In one embodiment, when the service interface of a worker node changes, the registry center notifies the management module to update the service interface information. This ensures that the service interfaces are up-to-date. Specifically, the management module can be notified through ZooKeeper's watcher mechanism.

[0053] In summary, the system proposed in the embodiments of the present invention has the following beneficial effects:

[0054] First, the control node integrates most of the workflow scheduling functions, including multiple modules. These modules work together to provide a good guarantee for workflow scheduling.

[0055] Secondly, the decision-making module of the control node provides a good basis for the distribution of workflow tasks; the web interface module provides a good workflow management interface for developers or managers, making it convenient to manage and maintain workflow tasks.

[0056] Third, it uses multiple nodes and is well integrated into the distributed framework, enabling excellent management and scheduling of distributed workflow tasks.

[0057] This invention also proposes a distributed workflow scheduling method, the principle of which is similar to that of a distributed workflow scheduling system, and will not be described in detail here.

[0058] Figure 5 The flowchart of the distributed workflow scheduling method in this embodiment of the invention includes:

[0059] Step 501: Receive the service interface information of the worker node. The service interface information of the worker node is sent by the registration center after registering the service interface of the worker node after receiving the service interface registration instruction of the worker node.

[0060] Step 502: Receive workflow tasks from at least one worker node;

[0061] Step 503: Determine the work node for executing each subtask based on the execution order and execution status of the subtasks in all workflow tasks;

[0062] Step 504: Send the subtask to the worker node through the worker node's service interface, whereby the worker node executes the subtask upon receiving it.

[0063] Step 505: After obtaining the execution result of the workflow task, feedback is sent through the service interface of the corresponding work node of the workflow task.

[0064] In one embodiment, after receiving a workflow task from at least one worker node, the method further includes:

[0065] Workflow tasks are broken down into subtasks, and the execution status and progress of all subtasks are managed.

[0066] After sending the subtask to the worker node through its service interface, the process also includes:

[0067] Upon receiving the execution result from the corresponding work node, the execution status of the subtask is recorded, and the next subtask to be executed is determined.

[0068] In one embodiment, the method further includes:

[0069] System information of registered nodes is collected periodically, including one or any combination of CPU utilization, network connection count, and memory utilization.

[0070] Based on the execution order and status of all subtasks in the workflow task, determine the work node for executing each subtask, including:

[0071] Based on the execution order and status of subtasks in all workflow tasks, as well as the system information of the registered nodes, determine the working node for executing each subtask.

[0072] In one embodiment, the method further includes:

[0073] Store system information, service interface information of worker nodes, execution order and execution status of subtasks.

[0074] In one embodiment, the method further includes: displaying the execution status and progress of workflow tasks.

[0075] In one embodiment, the method further includes:

[0076] Receive user click operation instructions, including subtask cancellation instructions, subtask trigger instructions, or subtask re-execution instructions;

[0077] When the click operation command is a subtask cancellation command, a subtask stop command is sent to the worker node executing the subtask; when the click operation command is a subtask trigger command, a subtask start command is sent to the worker node executing the subtask; when the click operation command is a subtask re-execution command, a subtask restart command is sent to the worker node executing the subtask.

[0078] In one embodiment, the method further includes:

[0079] The status of worker nodes is monitored through a heartbeat mechanism, and an alert message is generated when the status of a worker node is abnormal.

[0080] In one embodiment, the method further includes:

[0081] When receiving a service interface information update instruction from the registry center, update the service interface information.

[0082] In summary, the method proposed in the embodiments of the present invention has the following beneficial effects:

[0083] First, it completed the management of all processes for workflow scheduling, providing a good guarantee for workflow scheduling.

[0084] Secondly, it provides a good basis for the distribution of workflow tasks; and it provides a good workflow management interface for developers or managers, making it convenient to manage and maintain workflow tasks.

[0085] Third, it uses multiple nodes and is well integrated into the distributed framework, enabling excellent management and scheduling of distributed workflow tasks.

[0086] This invention also provides a computer device. Figure 6 This is a schematic diagram of a computer device in an embodiment of the present invention. The computer device 600 includes a memory 610, a processor 620, and a computer program 630 stored in the memory 610 and executable on the processor 620. When the processor 620 executes the computer program 630, it implements the above-mentioned distributed workflow scheduling method.

[0087] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described distributed workflow scheduling method.

[0088] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described distributed workflow scheduling method.

[0089] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program operating systems. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program operating system implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program business systems according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0091] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0092] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0093] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A distributed workflow scheduling system, characterized in that, include: At least one worker node is used to send service interface registration instructions to the registration node, submit workflow tasks to the management node, receive sub-tasks assigned by the management node and execute them. System information is periodically sent to the control node, including one or any combination of CPU utilization, network connection count, and memory utilization. The registration node is used to register the service interface of the worker node after receiving the service interface registration instruction from the worker node, and to send the registered service interface information of the worker node to the management node. The control node is used to receive service interface information from the worker nodes; receive workflow tasks from at least one worker node; determine the worker node to execute each subtask based on the execution order and execution status of the subtasks in all workflow tasks; and send the subtask to the worker node through the service interface of that worker node. After obtaining the execution result of the workflow task, feedback is sent through the service interface of the corresponding work node of the workflow task; Control nodes include: The management module is used to break down workflow tasks into subtasks and manage the execution status and progress of all subtasks. The decision module is used to determine the work node for executing each subtask based on the execution order and execution status of the subtasks in all workflow tasks; The communication module is used to communicate with the registered node and the worker node, receive service interface information from the worker node; receive workflow tasks from at least one worker node; send subtasks to the worker node through the worker node's service interface; and provide feedback on the execution results of the workflow tasks through the service interface of the corresponding worker node. The execution module is used to send the subtasks to be executed to the corresponding worker nodes through the communication module according to the execution order and execution status of the subtasks; upon receiving the execution result from the corresponding worker node, it records the execution status of the subtask and determines the next subtask to be executed. The system information collection module is used to collect system information of registered nodes through the communication module; The decision module is also used to: determine the working node for executing each subtask based on the execution order and execution status of subtasks in all workflow tasks, as well as the system information of the registered nodes; The storage module is used to store system information, service interface information of worker nodes, execution order and execution status of subtasks.

2. The system as described in claim 1, characterized in that, The control node also includes a web interface module, which displays the execution status and progress of workflow tasks.

3. The system as described in claim 1, characterized in that, The web interface module is also used to: receive user click operation instructions, the click operation instructions including subtask cancellation instructions, subtask trigger instructions, or subtask re-execution instructions; The communication module is also used to: send a subtask stop command to the worker node executing the subtask when the click operation command is a subtask cancellation command; send a subtask start command to the worker node executing the subtask when the click operation command is a subtask trigger command; and send a subtask restart command to the worker node executing the subtask when the click operation command is a subtask re-execution command.

4. The system as described in claim 1, characterized in that, The control node also includes a worker node status management module, which is used to monitor the status of worker nodes through a heartbeat mechanism and generate reminder information when the status of a worker node is abnormal.

5. The system as described in claim 1, characterized in that, The control node integrates a workflow engine.

6. The system as described in claim 1, characterized in that, The registry center is ZooKeeper.

7. The system as described in claim 1, characterized in that, When the service interface of a worker node changes, the registration center notifies the management module to update the service interface information.

8. A distributed workflow scheduling method, characterized in that, include: Receive the service interface information of the worker node. The service interface information of the worker node is sent by the registry center after registering the service interface of the worker node after receiving the service interface registration instruction of the worker node. Receive workflow tasks from at least one worker node; Based on the execution order and execution status of the subtasks in all workflow tasks, determine the work node for executing each subtask; The subtask is sent to the worker node through the worker node's service interface, and the worker node executes the subtask upon receiving it. After obtaining the execution result of the workflow task, feedback is provided through the service interface of the corresponding work node of the workflow task; After receiving a workflow task from at least one work node, the process also includes: splitting the workflow task into subtasks and managing the execution status and progress of all subtasks. After sending the subtask to the worker node through the worker node's service interface, the process also includes: upon receiving the execution result from the corresponding worker node, recording the execution status of the subtask, and determining the next subtask to be executed. It also includes: periodically collecting system information of registered nodes, wherein the system information includes one or any combination of CPU utilization, network connection count and memory utilization; Based on the execution order and execution status of subtasks in all workflow tasks, determine the working node for executing each subtask, including: based on the execution order and execution status of subtasks in all workflow tasks, and the system information of the registered node, determine the working node for executing each subtask. It also includes: storage system information, service interface information of worker nodes, execution order and execution status of subtasks.

9. The method as described in claim 8, characterized in that, Also includes: Display the execution status and progress of workflow tasks.

10. The method as described in claim 8, characterized in that, Also includes: Receive user click operation instructions, including subtask cancellation instructions, subtask trigger instructions, or subtask re-execution instructions; When the click operation command is a subtask cancellation command, a subtask stop command is sent to the worker node executing the subtask; when the click operation command is a subtask trigger command, a subtask start command is sent to the worker node executing the subtask; when the click operation command is a subtask re-execution command, a subtask restart command is sent to the worker node executing the subtask.

11. The method as described in claim 8, characterized in that, Also includes: The status of worker nodes is monitored through a heartbeat mechanism, and an alert message is generated when the status of a worker node is abnormal.

12. The method as described in claim 8, characterized in that, Also includes: When receiving a service interface information update instruction from the registry center, update the service interface information.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 8 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 8 to 12.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method according to any one of claims 8 to 12.

Images