An ordered message preserving system and its control method

Abstract

The present invention provides an ordered message preservation system and its control method, including: an asynchronous task processor, which groups tasks according to the document number, processes the tasks in the local task table in accordance with a preset node order, and sends the current node message corresponding to the task; characterized in that: before sending the current node message corresponding to the task, the asynchronous task processor checks whether all the previous node messages have been successfully sent: if there are previous node tasks that have not been successfully sent, the current task processing is suspended and delayed retry is triggered; otherwise, the MQ client is called to send the node message; by sending the node message to the asynchronous task processor and persistently storing it in the local task table, combined with the previous node status verification mechanism, it is ensured that the messages can be strictly sent in the preset node order. This method avoids blocking the main business process while ensuring the orderliness of the messages, is applicable to high-concurrency scenarios such as e-commerce and logistics, and has a wide range of applications.

Description

Technical Field

[0001] The present invention relates to the technical field of the distributed system sequential message order preservation technology for local task queues, and specifically relates to a sequential message order preservation system and its control method. Background Art

[0002] In business systems such as orders and logistics, there are usually multi-node state transitions. After each node's state changes, sequential messages need to be sent to the MQ client in a timely manner for downstream systems to process in a timely manner. In the prior art, message sending and business state changes are usually synchronous operations, and there are the following defects:

[0003] 1) Message sending failure leads to disorder: If the message sending of a certain node fails, the messages of subsequent nodes may be sent to the MQ client in advance due to the retry mechanism, thus destroying the order of tasks.

[0004] 2) Blocking the business process: If the synchronous sending of node messages fails, it will directly block the node state transition, thereby reducing the system throughput.

[0005] 3) Lack of global order control: Relying on the MQ's own sequential message mechanism (such as RocketMQ sequential messages) can only ensure the order within a single queue, but cannot achieve global order guarantee across node states.

[0006] MQ refers to the Message Queue, which is a container for storing messages and is essentially a queue.

[0007] Therefore, there is an urgent need for a new solution to solve the defects and deficiencies existing in the above prior art. Summary of the Invention

[0008] In order to solve the defects and deficiencies existing in the prior art, the present invention provides a sequential message order preservation system and its control method.

[0009] The specific solution provided by the present invention is as follows:

[0010] A sequential message order preservation system, the system includes:

[0011] A business service module, which drives the task node state to flow according to a preset node order, and when the task node state changes, generates a corresponding node message and persistently stores it in the local task table, marking it as the to-be-sent state;

[0012] A local task table, which persistently stores the tasks to be sent and records the current state of the tasks in real time;

[0013] An asynchronous task processor that groups tasks by document number, processes tasks in a local task table in a preset node order, and sends a current node message corresponding to the task;

[0014] Its characteristics are:

[0015] Before sending the current node message corresponding to the task, the asynchronous task processor verifies whether all previous node messages have been sent successfully:

[0016] If there is a previous node task whose previous node message was not sent successfully, the current task processing will be suspended and a delayed retry will be triggered;

[0017] If there is no predecessor node task whose predecessor node message has not been sent successfully, the MQ client is called to send the node message.

[0018] As a further preferred embodiment of the present invention, the preset node sequence is: review → delivery → receipt → completion.

[0019] As a further preferred embodiment of the present invention, the local task table is established according to the following steps:

[0020] 1) Create a database table;

[0021] 2) Add a joint index for the fields corresponding to the document number and node type.

[0022] As a further preferred embodiment of the present invention, the asynchronous task processor verifies whether all preceding node messages have been sent successfully by querying a local task table.

[0023] As a further preferred embodiment of the present invention, the asynchronous task processor verifies whether all preceding node messages have been sent successfully, including:

[0024] 1) Determine the types of all preceding nodes of the current node according to the preset node order;

[0025] 2) Query the task status of the previous node under the same document number in the local task table. If there is an unsuccessful task, it is determined to have failed the verification.

[0026] As a further preferred embodiment of the present invention, the delayed retry adopts an exponential backoff strategy, and the retry interval is 2n*base_delay,

[0027] in,

[0028] n is the number of retries, n=0, 1, 2...;

[0029] base_delay is the preset retry time.

[0030] As a further preferred embodiment of the present invention, when calling the MQ client to send a node message if there is no previously failed node task:

[0031] If the sending is successful, update the task status to successful;

[0032] If the sending fails, update the retry count and trigger the retry policy.

[0033] As a further preferred embodiment of the present invention, the system further includes a retry and circuit breaker control module, which manages the retry policy and circuit breaker rules for failed tasks.

[0034] As a further preferred embodiment of the present invention, in the retry and circuit breaker control module, if the task retry count exceeds the threshold, mark the task as failed and trigger the circuit breaker rule, and the circuit breaker rule includes:

[0035] 1) Send a circuit breaker alarm to the user;

[0036] 2) Pause processing of all subsequent tasks under the same document number until manual intervention for recovery.

[0037] Furthermore, the present invention also provides a control method for an ordered message ordering system, characterized by including the following steps:

[0038] S1: The business node status changes, generating a corresponding node message and persistently storing it in the local task table, marking it as the to-be-sent status;

[0039] S2: The asynchronous task processor groups the tasks by document number, processes the tasks in the local task table, and sends the current node message corresponding to the task;

[0040] S3: Before the asynchronous task processor sends the current node message corresponding to the task, verify whether all the previous node messages have been successfully sent;

[0041] If there is a previously failed node task, pause the current task processing and trigger a delayed retry;

[0042] If there is no previously failed node task, call the MQ client to send the node message;

[0043] S4: If the sending is successful, update the task status to successful;

[0044] If the sending fails, update the retry count and trigger the retry policy;

[0045] S5: If the task retry count exceeds the threshold, mark the task as failed and trigger the circuit breaker rule.

[0046] Compared with the prior art, the technical effects that the present invention can achieve include:

[0047] 1) The present invention provides an ordered message preservation system and its control method. By sending MQ messages to an asynchronous task processor and persistently storing them in a local task table, combined with a pre-order node status verification mechanism, it ensures that messages can be sent strictly in the preset node order. This method avoids blocking the main business process while ensuring the orderliness of messages, and is applicable to high-concurrency scenarios such as e-commerce and logistics, with a wide range of applications.

[0048] 2) The present invention provides an ordered message preservation system and its control method. By setting up a local task table and adopting an order verification mechanism: by persistently storing sending tasks and based on the document number and node order to check whether the pre-order node messages are sent successfully, it ensures that messages are sent strictly in order.

[0049] 3) The present invention provides an ordered message preservation system and its control method. By setting up an asynchronous task processor, the decoupling of business state flow and message sending is achieved. After the node state changes, it is immediately advanced, while the message sending is processed by the asynchronous task processor, avoiding blocking the main process.

[0050] 4) The present invention provides an ordered message preservation system and its control method, adopting a retry strategy and a circuit breaker rule: combining an exponential backoff strategy and a circuit breaker mechanism to balance reliability and system load. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The following shows the logical structure diagram of the system provided by the present invention;

[0052] Figure 2 The following shows the step flow chart of the method provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0054] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "front end", "back end", "both ends", "one end", "the other end", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0055] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] [First Embodiment]

[0057] As Figure 1 shown in the first embodiment provided by the present invention, this embodiment provides an ordered message preservation system, and the system includes:

[0058] A business service module, which drives the task node status to flow according to a preset node order. The preset node order in this embodiment is: review → outbound → receipt → completion. According to different task types, the preset node order can also be set to other required orders; when the task node status changes, a corresponding node message is generated and persistently stored in the local task table, and it is marked as the pending state.

[0059] A local task table, which is used to persistently store the tasks to be sent and record the current status of the tasks in real time; the local task table is established according to the following steps:

[0060] 1) Create a database table; the example code is as follows:

[0061] mq_send_task

[0062] CREATE TABLE mq_send_task (

[0063] id BIGINT AUTO_INCREMENT PRIMARY KEY,

[0064] order_id VARCHAR(64) NOT NULL, -- Document number

[0065] node_type VARCHAR(32) NOT NULL, -- Node type (e.g., "SIGNED")

[0066] status ENUM('pending','success','failed') DEFAULT 'pending',

[0067] retry_count INT DEFAULT 0,

[0068] created_time DATETIME NOT NULL );

[0070] 2) Add a composite index for the fields corresponding to the document number and node type, that is, add a composite index for the order_id and node_type fields to accelerate the subsequent sequential query process;

[0071] An asynchronous task processor that groups tasks by document number, processes tasks in the local task table according to the preset node order, and sends the current node message corresponding to the task;

[0072] The outstanding improvement of this embodiment compared with the prior art lies in:

[0073] Before the asynchronous task processor sends the current node message corresponding to the task, it checks whether all the previous node messages have been successfully sent: The asynchronous task processor can check whether all the previous node messages have been successfully sent by querying the local task table:

[0074] If there are previous node tasks for which the previous node messages have not been successfully sent, suspend the current task processing and trigger a delayed retry;

[0075] If there are no previous node tasks for which the previous node messages have not been successfully sent, call the MQ client to send the node message.

[0076] The logic code example of the asynchronous task processor in this embodiment is as follows:

[0077] / / Code example

[0078] public class AsyncTaskProcessor {

[0079] / / Message topic configuration

[0080] private CommonEventProvider commonEvent;

[0081] / / Process tasks grouped by document number

[0082] public void processTasks( ) {

[0083] List <mqtask>tasks = queryPendingTasks( );

[0084] tasks.groupBy(task ->task.getOrderId( ))

[0085] .forEach((orderId, taskGroup) ->{

[0086] / / Sort in node order (review → outbound → receipt → completion)

[0087] sortByNodeOrder(taskGroup);

[0088] for (MqTask task : taskGroup) {

[0089] if (checkPreTasksCompleted(task)){

[0090] sendMsgOrderly(task,MqUtil.getDestination(commonEvent.getTopic(),commonEvent.getTag( )),true);

[0091] } else {

[0092] delayRetry(task);

[0093] }

[0094] }

[0095] });

[0096] }

[0097] / / Check if previous nodes are completed

[0098] private boolean checkPreTasksCompleted(MqTask task) {

[0099] List <string>preNodeTypes = getPreNodeTypes(task.getNodeType());

[0100] return preNodeTypes.stream( )

[0101] .allMatch(preNode ->

[0102] taskDao.existsSuccessTask(task.getOrderId(),preNode));

[0103] }

[0104] }

[0105] / / Send sequential messages. Under the premise of node order, messages should also be sent sequentially

[0106] private void sendMsgOrderly(MqTask payload, String destination,boolean orderly) {

[0107] String msgId = payload.getId( );

[0108] String hashKey = payload.getOrderId( );

[0109] RocketMQTemplate rocketMqTemplate = (RocketMQTemplate)this.rocketMqTemplateObjectProvider.getIfAvailable( );

[0110] Assert.notNull(rocketMqTemplate, "rocketMQTemplate is null");

[0111] Message<>message = MqDtxUtil.buildRocketMqMessage(payload);

[0112] log.info("send message, body: destination={},message={}", newObject[ ]{destination, message});

[0113] try {

[0114] SendResult sendResult;

[0115] if (orderly) {

[0116] sendResult = rocketMqTemplate.syncSendOrderly(destination, message,hashKey, 10000L);

[0117] } else {

[0118] sendResult = rocketMqTemplate.syncSend(destination,message, 10000L);

[0119] }

[0120] if (SendStatus.SEND_OK == sendResult.getSendStatus( )) {

[0121] log.info("send success,sendResult={}", sendResult);

[0122] this.updateSubTaskStatus(SubMsgTaskStatus.SEND_SUCCESS);

[0123] } else {

[0124] log.error("sendfail,sendResult:{}", sendResult);

[0125] this.updateSubTaskStatus(SubMsgTaskStatus.SEND_FAIL);

[0126] }

[0127] } catch (Exception var10) {

[0128] log.error("Occur an Exception when innerSendMessage( )",var10);

[0129] this.updateSubTaskStatus(SubMsgTaskStatus.SEND_FAIL);

[0130] }

[0131] }

[0132] The asynchronous task processor checks whether all the messages of the previous nodes have been successfully sent, including:

[0133] 1) Determine all the previous node types of the current node according to the preset node order;

[0134] 2) Query the task status of the previous nodes under the same document number in the local task table. If there are unsuccessfully completed tasks, it is determined that the check fails; Therefore, adding a composite index to the fields corresponding to the document number and node type can effectively facilitate the accelerated sequential query process here.

[0135] In this embodiment, exponential backoff strategy is adopted for delayed retry, and the retry interval is 2n * base_delay, where,

[0136] n is the number of retries, n = 0, 1, 2...;

[0137] base_delay is the preset retry time,

[0138] For example, base_delay can be set to 2s, then the retry time intervals are as follows:

[0139] The first time: 20 * 2s = 2s;

[0140] The second time: 21 * 2s = 4s;

[0141] The third time: 22 * 2s = 8s;

[0142] ...;

[0143] And so on until the retry is successful,

[0144] Thus, by adopting the exponential backoff strategy for delayed retry, the reliability and system load are balanced.

[0145] If there are no unsuccessfully sent previous node tasks, when calling the MQ client to send node messages:

[0146] If the sending is successful, update the task status to successful;

[0147] If the sending fails, update the number of retries and trigger the retry strategy.

[0148] The triggered retry policy is the aforementioned exponential backoff policy; by updating the retry count to record in real time the current number of retries, so that at the next execution, according to the current number of executions, the next execution time is calculated through the formula: 2^n * base_delay. Therefore, it not only checks whether the previous node message is successfully sent based on the document number and node order, but also considers the impact of the success or failure of this message sending on the delayed retry time to ensure the strict sequential sending of messages.

[0149] The system in this embodiment further includes a retry and circuit breaker control module, and the retry and circuit breaker control module manages the retry policy and circuit breaker rules for failed tasks. Specifically, in this embodiment, in the retry and circuit breaker control module, if the task retry count exceeds the threshold, the task is marked as failed and the circuit breaker rule is triggered. The circuit breaker rule includes:

[0150] 1) Send a circuit breaker alarm to the user; and

[0151] 2) Pause processing all subsequent tasks under the same document number until manual intervention for recovery.

[0152] [Second Embodiment]

[0153] The second embodiment of the present invention further provides a control method for the sequential message ordering system mentioned in the first embodiment, including the following steps:

[0154] S1: The business node status changes, generating a corresponding node message and persistently storing it in the local task table, marking it as the to-be-sent status;

[0155] S2: The asynchronous task processor groups the tasks by document number, processes the tasks in the local task table, and sends the current node message corresponding to the task;

[0156] S3: Before the asynchronous task processor sends the current node message corresponding to the task, check whether all the previous node messages have been successfully sent;

[0157] If there is a previous node task with a previous node message not successfully sent, pause the current task processing and trigger a delayed retry;

[0158] If there is no previous node task with a previous node message not successfully sent, call the MQ client to send the node message;

[0159] S4: If the sending is successful, update the task status to successful;

[0160] If the sending fails, update the retry count and trigger the retry policy;

[0161] S5: If the task retry count exceeds the threshold, mark the task as failed and trigger the circuit breaker rule.

[0162] It is obvious to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, in all respects, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.< / string> < / mqtask>

Claims

1. A sequential message order-preserving system, the system comprising: A business service module that drives the task node status to flow according to a preset node order, and when the task node status changes, generates a corresponding node message and persists it in the local task table, marking it as the to-be-sent status; A local task table that persistently stores the tasks to be sent and records the current status of the tasks in real time; An asynchronous task processor that groups tasks by document number, processes the tasks in the local task table according to the preset node order, and sends the current node message corresponding to the task; It is characterized in that: Before the asynchronous task processor sends the current node message corresponding to the task, it verifies whether all the previous node messages have been successfully sent: If there is a previous node task with a previous node message that has not been successfully sent, suspend the current task processing and trigger a delayed retry; If there is no previous node task with a previous node message that has not been successfully sent, call the MQ client to send the node message.

2. The sequential message order-preserving system according to claim 1, wherein: The preset node order is: review → outbound → receipt → completion.

3. The sequential message order-preserving system according to claim 1, wherein: The local task table is established according to the following steps: 1) Create a database table; 2) Add a composite index to the fields corresponding to the document number and node type.

4. The sequential message order-preserving system according to claim 1, wherein: The asynchronous task processor verifies whether all the previous node messages have been successfully sent by querying the local task table.

5. The sequential message ordering system according to claim 1, wherein: The asynchronous task processor verifies whether all the previous node messages have been successfully sent, including: 1) Determine all the previous node types of the current node according to the preset node order; 2) Query the task status of the previous nodes under the same document number in the local task table. If there is an unsuccessful task, it is determined that the verification fails.

6. The sequential message order-preserving system according to claim 1, wherein: The delayed retry adopts an exponential backoff strategy, and the retry interval is 2n * base_delay, where, n is the number of retries, n = 0, 1, 2...; base_delay is the preset retry time.

7. The sequential message order-preserving system according to claim 6, wherein: If there is no previous node task with an unsuccessful send, when calling the MQ client to send the node message: If the send is successful, update the task status to successful; If the send fails, update the number of retries and trigger the retry strategy.

8. The sequential message ordering system according to claim 1, characterized in that: The system further includes a retry and circuit breaker control module that manages the retry strategy and circuit breaker rules for failed tasks.

9. The sequential message order-preserving system according to claim 8, wherein: In the retry and circuit breaker control module, if the number of task retries exceeds the threshold, mark the task as failed and trigger the circuit breaker rules, and the circuit breaker rules include: 1) Send a circuit breaker alarm to the user; 2) Suspend processing all subsequent tasks under the same document number until manual intervention resumes.

10. A control method for an ordered message preservation system according to any one of claims 1-9, characterized in that: Including the following steps: S1: The business node status changes, generates a corresponding node message and persists it in the local task table, marking it as the to-be-sent status; S2: The asynchronous task processor groups tasks by document number, processes the tasks in the local task table, and sends the current node message corresponding to the task; S3: Before the asynchronous task processor sends the current node message corresponding to the task, verify whether all the previous node messages have been successfully sent; If there are pre - node tasks that have not been successfully sent, suspend the current task processing and trigger a delayed retry; If there are no pre - node tasks that have not been successfully sent, call the MQ client to send node messages; S4: If the sending is successful, update the task status to successful; If the sending fails, update the retry count and trigger the retry policy; S5: If the task retry count exceeds the threshold, mark the task as failed and trigger the circuit - breaker rule.

Images