Business process driving engine system realized based on message queue and operation method

Abstract

The invention discloses a business process driving engine system and an operation method implemented based on a message queue, and relates to the technical field of business process processing, the business process driving engine system comprises a message generator for decomposing a process into more than one subtask and creating a corresponding message for each subtask; the message queues comprise different message queues for identifying task types; the task processor is used for acquiring the message to be processed from the message queue and executing a corresponding sub-task; the process engine is used for selecting a corresponding task processor after the message queue monitors the message, and delivering the message to the task processor for processing; and receiving a feedback task result, and creating a message of a corresponding next sub-task in combination with a current sub-task processing result and the sub-task of the subsequent process to realize circular processing. According to the business process driving engine system based on the message queue, the response speed of the process engine is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of business process management, and in particular to a business process driving engine system and an operation method based on a message queue. Background Art

[0002] With the continuous development of business process management (BPM), BPMN (Business Process Model and Notation) 2.0 has become a widely accepted standard in the industry. However, when implementing complex business processes, traditional BPMN2.0 process engines may face problems such as low processing efficiency, limited fault tolerance, poor scalability, and single point failure. Specifically, the problem of low processing efficiency is that existing process engines often encounter performance bottlenecks when processing large-scale business processes. Since the process engine needs to process a large amount of tasks and data, the execution speed is slowed down and cannot meet the real-time requirements.

[0003] The problem of limited fault tolerance is that the process engine lacks a flexible fault-tolerant processing mechanism when facing abnormal situations. Once an error or abnormality occurs, the process engine often cannot automatically roll back or continue execution, and manual intervention is required, which affects the continuity and stability of the business.

[0004] The problem of poor scalability is that when the existing process engine is connected to a new business scenario or when the existing business is expanded, it often requires large-scale changes and customized development. Such poor scalability cannot meet the rapidly changing needs of enterprises, increasing the difficulty and cost of development and maintenance.

[0005] The problem of lack of flexibility is that the existing process engines are relatively rigid when defining process rules and executing processes. In different business scenarios, process rules often need to be customized according to specific needs, and process engines that lack flexibility cannot meet this requirement. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a BPMN process driving engine system based on a message queue, aiming to improve the response speed of the process engine.

[0007] In order to solve the above technical problems, the present invention is solved by the following technical solutions:

[0008] A business process driving engine system based on message queue implementation includes the following structure:

[0009] A message generator that decomposes the process into one or more subtasks and creates a corresponding message for each subtask;

[0010] Message queues, including different message queues that identify task types;

[0011] The task processor obtains the pending messages from the message queue and executes the corresponding subtasks;

[0012] The process engine is used to listen to the message queue after receiving the message, generate the node and path information required for the process execution according to the message, select the corresponding task processor in combination with the message content, and deliver the message to the task processor for processing; receive the feedback of the task result, determine whether the task result is correct, if correct, create the corresponding next subtask message in combination with the current subtask processing result and the subtask of the subsequent process, and pass it to the message queue for processing; if it is an error, re-call the subtask that failed to be processed and pass it to the message queue for processing.

[0013] Optionally, the process engine includes a process file storage module, a process file parsing module and a process execution module, wherein:

[0014] The process file storage module reads the process file and submits it to the process file parsing module;

[0015] The process file parsing module parses the process definition file stored in the process definition library and generates the node and path information required for process execution;

[0016] The process execution module drives the execution of the process based on the node and path information generated by the process parsing module and the messages in the message queue.

[0017] Optionally, the process engine also includes:

[0018] The context storage module stores the control data and business data generated during the process operation.

[0019] Optionally, the process engine also includes: a management module for process file publishing and version update management.

[0020] Optionally, the process engine also includes a process recording module for recording tracking records during process operation.

[0021] Optionally, the message queue includes a fast processing group and a slow processing group, wherein:

[0022] The quick processing group is used for the consumption and delivery of common process documents;

[0023] The slow processing group is used to handle tasks that take a long time.

[0024] Optionally, the process file is a file based on the BPMN2.0 workflow protocol.

[0025] A method for operating a business process driving engine comprises the following steps:

[0026] Step 1, the process engine calls the message generator;

[0027] Step 2: Decompose the process file into more than one subtask and create a corresponding message for each subtask;

[0028] Step 3: Different message queues that identify the task type obtain messages of the corresponding tasks and deliver the messages according to the first-in-first-out principle;

[0029] Step 4: After the process engine monitors the message, it generates the node and path information required for process execution according to the message, combines the message content, selects the corresponding task processor, and delivers the message to the task processor for processing;

[0030] Step 5: After the task processor completes the task, it will feed back the task result to the process engine to determine whether the task result is correct. If it is correct, it will return to step 1, create a corresponding next subtask message based on the current subtask processing result and the subtask of the subsequent process, and pass it to the message queue for processing;

[0031] If an error occurs, return to step 1 to re-call the subtask that failed to be processed and pass it to the message queue for processing.

[0032] Optionally, when the number of failures of the same subtask exceeds a threshold, the task is marked as failed.

[0033] Optionally, multiple copies are used to implement the execution of the method from step 1 to step 5, allowing the process engines of the multiple copies to listen to the same message queue.

[0034] Beneficial effects: Existing message queues all serve as a simple delivery function. The present invention proposes to use message queues as the core driving capability to automatically drive BPMN processes, combining the BPMN core process orchestration capability with message queues. By configuring different types of message queues, subtasks of the same type parsed in different process files are placed in the same message queue for execution. Multiple copies can execute subtasks of the same type, and the ability to execute different types of subtasks at the same time can be achieved. In this way, the processing efficiency of the process and the throughput of the system are improved through asynchronous processing and parallel execution of tasks. Compared with the synchronous execution method, which requires all subtasks parsed in the same process file to be completed before the next set of tasks can be processed, the message queue-based method proposed in the present invention is stable and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 It is a framework diagram of the business process driving engine system;

[0037] Figure 2 It is a flowchart of the method of operating the business process driving engine;

[0038] Figure 3 It is a BPMN file storage and parsing flowchart;

[0039] Figure 4 It is a BPMN process execution diagram;

[0040] Figure 5 is the message model of the present invention;

[0041] Figure 6 is the message delivery model of the present invention;

[0042] Figure 7 It is the queue model of the present invention. DETAILED DESCRIPTION

[0043] The present invention is further described in detail below in conjunction with embodiments. The following embodiments are for explanation of the present invention but the present invention is not limited to the following embodiments.

[0044] Embodiment 1: A business process driving engine system implemented based on a message queue includes the following structure:

[0045] A message generator that decomposes the process into one or more subtasks and creates a corresponding message for each subtask;

[0046] Message queues, including different message queues that identify task types;

[0047] The task processor obtains the pending messages from the message queue and executes the corresponding subtasks;

[0048] The process engine is used to listen to the message queue after receiving the message, parse the BPMN process definition file, generate the node and path information required for process execution, select the corresponding task processor based on the message content, and deliver the message to the task processor for processing; receive the feedback of the task result, and judge whether the task result is correct according to the BPMN process definition file. If correct, create the corresponding next subtask message based on the current subtask processing result and the subtask of the subsequent process, and pass it to the message queue for processing; if it is wrong, re-call the subtask that failed to be processed and pass it to the message queue for processing.

[0049] Among them, the message generator and task processor are both included in the adapter.

[0050] After receiving the message from the message queue, the task processor submits the subtask to different task processing units for processing according to the tag contained in the message.

[0051] For example: after receiving a process file, the message generator generates different subtasks (such as the four tasks below) and generates corresponding messages.

[0052] The existing initialization task->download task->screenshot task->transcoding task are considered as four subtasks parsed from a process file. The task processor receives the message of the initialization task, submits the initialization task to the initialization task processing unit for processing, and submits the processing result to the process engine after the processing is completed. The process engine determines the processing result: if it is a correct result, the message will be reassembled and the mark will be set as "download task", and then the message will be delivered to the message queue; if it is an incorrect result, it will be retried. If the retry still fails, the process will be marked as failed.

[0053] After receiving the message carrying the mark of the download task, the task processor submits the task to the download processing unit, and the subsequent process is executed similarly to the above.

[0054] like Figure 6 and Figure 7 As shown, the green queues (queue 1 and queue 4) are the corresponding message queues for the initialization task, the purple queues (queue 2 and queue 5) are the corresponding message queues for the screenshot task, and the orange queues (queue 3 and queue 6) are the corresponding message queues for the transcoding task.

[0055] Figure 6 In the message production stage, i.e., the message generator, the received process file is decomposed into one or more subtasks, and a corresponding message is created for each subtask. Each created message carries a tag, such as an initialization tag, a screenshot tag, and a transcoding tag.

[0056] Messages with different tags are delivered to the corresponding message queues. Multiple message queues of the same type are allowed to exist, such as queue 1 and queue 4, queue 2 and queue 5, queue 3 and queue 6 in the figure, to meet the needs of parallel processing of multiple tasks of the same type and speed up system processing time.

[0057] There are four subtasks: initialization task->download task->screenshot task->transcoding task. The processing order of these four subtasks is configured to be processed in sequence. After the process engine determines that the current subtask is completed, it reassembles the completion result of the current subtask and the next subtask message and delivers them to the message queue. The messages in the message queue wait to be delivered to the corresponding task processor according to the first-in-first-out principle.

[0058] As a preferred solution, by defining different process rules and processing logic, it can adapt to different types of business processes and support dynamic process modification and adjustment. For example, if the process rules and processing logic are defined as the screenshot task as the last task to process the process, the four subtasks initialization task -> download task -> transcoding task -> screenshot task will generate messages according to the corresponding rules and deliver them to the message queue in turn for processing.

[0059] The operation of the process engine mentioned above is handled separately by its different modules, refer to Figure 3 and Figure 4 As shown, the details are as follows: the process engine includes a process file storage module, a process file parsing module and a process execution module, wherein the process file storage module reads the process file and submits it to the process file parsing module.

[0060] The process file parsing module parses the process definition file stored in the process definition library and generates the node and path information required for process execution.

[0061] The process execution module drives the execution of the process by calling corresponding services or components according to the node and path information generated by the process parsing module and the messages in the message queue module. The execution of the driving process includes 1) delivering the message to the corresponding task processor for processing; 2) receiving the task result and calling the message generator to generate the next task message according to the node and path information required for the process execution.

[0062] This application is a driving engine based on the BPMN (Business Process Model and Notation) 2.0 protocol. The process file storage module is responsible for reading the BPMN2.0 file and submitting it to the parser; the process file parsing module parses the process definition file stored in the process definition library according to the BPMN2.0 specification and generates the node and path information required for process execution.

[0063] As an example, a BPMN2.0 parser (such as Camunda BPM, Activiti, etc.) is used to parse the BPMN2.0 process definition file stored in the process definition library to generate the node and path information required for process execution.

[0064] Among them, the message is encapsulated / assembled through the following message model, refer to Figure 5 ,The message model includes: version number, serial number, execution time, timeout, end time, number of retries, business data, executed node set, unexecuted node set, end tag, message tag, and message subject.

[0065] As a solution to enhance functionality, the process engine also includes: a context storage module to store control data and business data generated during process operation. Preferably, a file database (Mysql) or a memory database (Redis) is used to store various data during process execution.

[0066] As a solution to enhance functionality, the process engine further includes: a management module for process file publishing and version update management. For example, the module is responsible for the publishing and version management of BPMN2.0 files. Accordingly, the method for driving process execution by the process execution module further includes 3) loading context initial data and writing it into the context data storage module.

[0067] As a solution to enhance functionality, the process engine also includes a process recording module for recording tracking records during process operation. The process engine also has the function of managing and monitoring the process status, and can track and record the execution of the process. For example, a file database (Mysql) is used to record various events and exceptions during the process execution. Accordingly, the method of driving process execution by the process execution module also includes 4) writing the tracking records into the process recording module.

[0068] As a preferred solution, the message queue includes a fast processing group and a slow processing group, refer to Figure 7 As shown, the fast processing group is used to process the message delivery that takes a short time; the slow processing group is used to process the message delivery that takes a long time.

[0069] Generally speaking, the processes of the BPMN2.0 protocol are consumed and delivered in the fast processing group. Both the fast processing group and the slow processing group include several message queues. For example, the fast processing group is divided into 10 consumption queues, which can be divided according to specific business types. Tasks of the same type use the same queue, such as one queue for initializing data tasks and one queue for screenshot tasks.

[0070] The slow group is used to process time-consuming tasks, such as downloading tasks, transcoding tasks, etc. Dividing into fast processing groups and slow processing groups helps improve the overall task processing efficiency and prevent time-consuming tasks from blocking other tasks and becoming a bottleneck for the overall process.

[0071] Embodiment 2: A method for operating a business process driving engine is disclosed, comprising the following steps:

[0072] Step 1, the process engine calls the message generator;

[0073] Step 2: Decompose the process file into more than one subtask and create a corresponding message for each subtask;

[0074] Step 3: Different message queues that identify the task type obtain messages of the corresponding tasks and deliver the messages according to the first-in-first-out principle;

[0075] Step 4: After the process engine listens to the message, it parses the BPMN process definition file, generates the node and path information required for process execution, selects the corresponding task processor based on the message content, and delivers the message to the task processor for processing;

[0076] Step 5: After the task processor completes the task, it will feed back the task result to the process engine. The process engine will determine whether the task result is correct based on the BPMN process definition file. If it is correct, it will return to step 1, create a corresponding next subtask message based on the current subtask processing result and the subtask of the subsequent process, and pass it to the message queue for processing;

[0077] If an error occurs, return to step 1 to re-call the subtask that failed to be processed and pass it to the message queue for processing.

[0078] The message includes the information of the subtask, the identifier of the process to which it belongs, and metadata.

[0079] As a preferred solution, multiple copies are used to implement the operation of the method of step 1 to step 5, allowing the process engines of multiple copies to monitor the same message queue.

[0080] In a cluster environment, processors of different replicas can listen to the same message queue. When some replicas fail, other replicas can still schedule and process messages according to BPMN2.0, ensuring reliable execution and processing of tasks.

[0081] When the task fails to execute, it returns to step 1 to call the failed subtask again and pass it to the message queue for processing. This step is called the retry mechanism, which is specifically divided into automatic retry method and manual retry method.

[0082] Automatic retry method: After the task execution fails in the task processor, it is handed over to the process engine. The process engine makes a judgment based on the current number of failures. If it is less than the maximum number of failures, the message queue is notified. If the consumption fails this time, the message queue sends the same message to the task processor again to achieve the retry capability; if the maximum number of retries has been reached (the preset threshold parameter), no retries will be made and the task will be marked as failed.

[0083] The manual retry method includes: manually submitting the task mark to be retried to the process engine, the process engine will reassemble the message and send it to the message queue, the message queue will deliver the message to the task processor, and the task processor will consume the message processing task again.

[0084] As a preferred method, the system processing speed is increased by adding replicas or threads. Due to the independence of tasks, the system can be easily expanded horizontally to cope with the processing requirements of processes of different scales and loads, thus achieving the scalability of the system.

[0085] As a preferred method, if other systems load this system at the same time, cross-system interaction can be achieved. When the message parsed by the process engine carries the identifier of other systems, it can be delivered to the message queue of other systems. Realize collaborative execution of processes

[0086] The replica referred to in this application is a complete, independently run unit module in the business process management system, which includes the functions of the entire business process engine. This replica can be understood as an independent business process execution environment, which can parse and execute BPMN (Business Process Model and Notation) files, process the sending and receiving of messages, and execute tasks and other operations.

[0087] The message queue in the present application preferably adopts a high-performance message queue system (such as RocketMQ, RabbitMQ, etc.) to realize the reception, storage and forwarding of messages.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0089] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0090] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a readable storage medium. Based on such an understanding, an embodiment of the present invention also proposes a computer-readable storage medium, which can implement the method of any of the above-mentioned embodiments when the executable computer program in the storage medium is executed by a processor.

[0091] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the method of any of the above embodiments when executed by a processor.

[0092] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A business process driving engine system based on message queue, characterized in that: Includes the following structure: A message generator that decomposes the process into one or more subtasks and creates a corresponding message for each subtask; Message queues, including different message queues that identify task types; The task processor obtains the pending messages from the message queue and executes the corresponding subtasks; The process engine is used to monitor the message queue after receiving the message, generate the node and path information required for the process execution according to the message, select the corresponding task processor based on the message content, and deliver the message to the task processor for processing; Receive the feedback of the task result and determine whether the task result is correct. If correct, create the corresponding next subtask message based on the current subtask processing result and the subtask of the subsequent process, and pass it to the message queue for processing; if it is wrong, re-call the subtask that failed to be processed and pass it to the message queue for processing.

2. A business process driven engine system based on message queue implementation as claimed in claim 1, characterized in that: The process engine includes a process file storage module, a process file parsing module and a process execution module, among which, The process file storage module reads the process file and submits it to the process file parsing module; The process file parsing module parses the process definition file stored in the process definition library and generates the node and path information required for process execution; The process execution module drives the execution of the process based on the node and path information generated by the process parsing module and the messages in the message queue.

3. A business process driven engine system based on message queue implementation as claimed in claim 2, characterized in that: The process engine also includes: The context storage module stores the control data and business data generated during the process operation.

4. A business process driven engine system based on message queue implementation as claimed in claim 2, characterized in that: The process engine also includes: a management module for process file publishing and version update management.

5. A business process driven engine system based on message queue implementation as claimed in claim 2, characterized in that: The process engine also includes a process recording module for recording tracking records during process operation.

6. A business process driven engine system based on message queue implementation as claimed in claim 2, characterized in that: The message queue includes a fast processing group and a slow processing group, where: The quick processing group is used for the consumption and delivery of common process documents; The slow processing group is used to handle tasks that take a long time.

7. A business process driven engine system based on a message queue as claimed in any one of claims 1 to 6, characterized in that: The process file is a file based on the BPMN2.0 workflow protocol.

8. A method for operating a business process driven engine, characterized in that: The steps include: Step 1, the process engine calls the message generator; Step 2: Decompose the process file into more than one subtask and create a corresponding message for each subtask; Step 3: Different message queues that identify the task type obtain messages of the corresponding tasks and deliver the messages according to the first-in-first-out principle; Step 4: After the process engine monitors the message, it generates the node and path information required for process execution according to the message, combines the message content, selects the corresponding task processor, and delivers the message to the task processor for processing; Step 5: After the task processor completes the task, it will feed back the task result to the process engine to determine whether the task result is correct. If it is correct, it will return to step 1, create a corresponding next subtask message based on the current subtask processing result and the subtask of the subsequent process, and pass it to the message queue for processing; If an error occurs, return to step 1 to re-call the subtask that failed to be processed and pass it to the message queue for processing.

9. A method for operating a business process driven engine as claimed in claim 8, characterized in that: When the number of failures of the same subtask exceeds the threshold, the task is marked as failed.

10. The method for operating a business process driven engine according to claim 8, characterized in that: Multiple copies are used to implement the execution of the method from step 1 to step 5, allowing the process engines of multiple copies to listen to the same message queue.

Images