Sequence message order preserving system and control method thereof

Abstract

The invention provides a sequence message order preserving system and a control method thereof, and the system comprises an asynchronous task processor which carries out the grouping of tasks according to document numbers, processes the tasks in a local task table according to a preset node sequence, and transmits a current node message corresponding to the task; the method is characterized in that an asynchronous task processor verifies whether all preorder node messages are successfully sent before sending the current node messages corresponding to a task: if a preorder node task which is not successfully sent exists, current task processing is paused, and delay retry is triggered; otherwise, calling the MQ client to send a node message; according to the method, the node messages are sent to the asynchronous task processor and persistently stored in the local task table, a preorder node state verification mechanism is combined, it is ensured that the messages can be sent strictly according to the preset node sequence, and the method avoids blocking of the main service process while guaranteeing the message sequence, is suitable for high-concurrency scenes such as e-commerce and logistics, and is wide in application range.

Description

Technical Field

[0001] The present invention relates to the technical field of the distributed system sequential message ordering technology for local task queues, and particularly relates to a sequential message ordering system and its control method. Background Art

[0002] In business systems such as orders and logistics, there are usually multi-node state transitions. After each node's state changes, sequential messages need to be sent to the MQ client in a timely manner for downstream systems to process in a timely manner. In the prior art, message sending and business state changes are usually synchronous operations, and there are the following defects: 1) Message sending failure leads to disorder: If the message sending of a certain node fails, the messages of subsequent nodes may be sent to the MQ client in advance due to the retry mechanism, thus destroying the order of tasks.

[0003] 2) Blocking the business process: If the synchronous sending of node messages fails, it will directly block the node state transition, thus reducing the system throughput.

[0004] 3) Lack of global order control: Relying on the MQ's own sequential message mechanism (such as RocketMQ sequential messages) can only ensure the order within a single queue, but cannot achieve global order guarantee across node states.

[0005] MQ refers to Message Queue, which is a container for storing messages and is essentially a queue.

[0006] Therefore, there is an urgent need for a new solution to solve the defects and deficiencies existing in the above prior art. Summary of the Invention

[0007] In order to solve the defects and deficiencies existing in the prior art, the present invention provides a sequential message ordering system and its control method.

[0008] The specific solution provided by the present invention is as follows: A sequential message ordering system, the system includes: A business service module, which drives the task node state to flow according to a preset node order, and when the task node state changes, generates a corresponding node message and persistently stores it in the local task table, and marks it as the to-be-sent state; A local task table, which persistently stores the tasks to be sent and records the current state of the tasks in real time; An asynchronous task processor, which groups the tasks according to the document number, processes the tasks in the local task table according to the preset node order, and sends the current node message corresponding to the task; It is characterized in that: Before sending the current node message corresponding to the task, the asynchronous task processor verifies whether all the previous node messages have been successfully sent: If there are previous node tasks that have not been successfully sent, suspend the current task processing and trigger a delayed retry; If there are no previous node tasks that have not been successfully sent, call the MQ client to send the node message.

[0009] As a further preferred embodiment of the present invention, the preset node order is: review → outbound → receipt → completion.

[0010] As a further preferred embodiment of the present invention, the local task table is established according to the following steps: 1) Create a database table; 2) Add a composite index to the fields corresponding to the document number and node type.

[0011] As a further preferred embodiment of the present invention, the asynchronous task processor verifies whether all the previous node messages have been successfully sent by querying the local task table.

[0012] As a further preferred embodiment of the present invention, the asynchronous task processor verifies whether all the previous node messages have been successfully sent, including: 1) Determine all the previous node types of the current node according to the preset node order; 2) Query the task status of the previous nodes under the same document number in the local task table. If there are unsuccessful tasks, it is determined that the verification fails.

[0013] As a further preferred embodiment of the present invention, the delayed retry adopts an exponential backoff strategy, and the retry interval is 2 n * base_delay, where, n is the number of retries, n = 0, 1, 2...; base_delay is the preset retry time.

[0014] As a further preferred embodiment of the present invention, if there are no previous node tasks that have not been successfully sent, when calling the MQ client to send the node message: If the sending is successful, update the task status to successful; If the sending fails, update the number of retries and trigger the retry strategy.

[0015] As a further preferred embodiment of the present invention, the system further includes a retry and circuit breaker control module, and the retry and circuit breaker control module manages the retry strategy and circuit breaker rules for failed tasks.

[0016] As a further preferred embodiment of the present invention, in the retry and fuse control module, if the number of task retries exceeds the threshold, the task is marked as failed and the fuse rule is triggered. The fuse rule includes: 1) Send a fuse warning to the user; 2) Pause processing all subsequent tasks under the same document number until manual intervention for recovery.

[0017] Furthermore, the present invention also provides a control method for an ordered message ordering system, which is characterized by including the following steps: S1: The status of the service node changes, generating a corresponding node message and persistently storing it in the local task table, marking it as the to-be-sent status; S2: The asynchronous task processor groups tasks by document number, processes the tasks in the local task table, and sends the current node message corresponding to the task; S3: Before the asynchronous task processor sends the current node message corresponding to the task, verify whether all the previous node messages have been successfully sent; If there are previous node tasks that have not been successfully sent, pause the current task processing and trigger a delayed retry; If there are no previous node tasks that have not been successfully sent, call the MQ client to send the node message; S4: If the sending is successful, update the task status to successful; If the sending fails, update the number of retries and trigger the retry policy; S5: If the number of task retries exceeds the threshold, mark the task as failed and trigger the fuse rule.

[0018] Compared with the prior art, the technical effects that the present invention can achieve include: 1) The present invention provides an ordered message ordering system and its control method. By sending the MQ message to the asynchronous task processor and persistently storing it in the local task table, combined with the previous node status verification mechanism, it ensures that the message can be sent strictly in the preset node order. This method avoids blocking the main business process while ensuring the message orderliness, and is applicable to high-concurrency scenarios such as e-commerce and logistics, with a wide range of applications.

[0019] 2) The present invention provides an ordered message ordering system and its control method. By setting up a local task table and adopting an order verification mechanism: by persistently storing the sending task and implementing the check of whether the previous node message has been successfully sent based on the document number and node order, it ensures that the message is sent strictly in order.

[0020] 3) The present invention provides an ordered message preservation system and its control method. By setting up an asynchronous task processor, the business status transition and message sending are decoupled. After the node status changes, it is immediately advanced, and the message sending is processed by the asynchronous task processor to avoid blocking the main process.

[0021] 4) The present invention provides an ordered message preservation system and its control method, which adopts a retry strategy and a fusing rule: combining an exponential backoff strategy and a fusing mechanism to balance reliability and system load. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The following shows the logical structure diagram of the system provided by the present invention; Figure 2 The following shows the step flow diagram of the method provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "front end", "backend", "both ends", "one end", "the other end", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0025] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0026] [First Embodiment] As Figure 1 The following shows the first embodiment provided by the present invention. This embodiment provides an ordered message preservation system, and the system includes: Business service module, which drives the task node status to flow according to the preset node order. The preset node order in this embodiment is: review → outbound → receipt → completion. According to different task types, the preset node order can also be set to other required orders; when the task node status changes, corresponding node messages are generated and persistently stored in the local task table, and marked as the pending state. Local task table, which is used to persistently store tasks to be sent and record the current status of tasks in real time; the local task table is established according to the following steps: 1) Create a database table; the example code is as follows: mq_send_task CREATE TABLE mq_send_task ( id BIGINT AUTO_INCREMENT PRIMARY KEY, order_id VARCHAR(64) NOT NULL, -- Bill number node_type VARCHAR(32) NOT NULL, -- Node type (e.g., "SIGNED") status ENUM('pending','success', 'failed') DEFAULT 'pending', retry_count INT DEFAULT 0, created_time DATETIME NOT NULL ); 2) Add a composite index to the fields corresponding to the bill number and node type, that is, add a composite index to the order_id and node_type fields to accelerate the subsequent sequential query process; Asynchronous task processor, which groups tasks by bill number, processes tasks in the local task table according to the preset node order, and sends the current node message corresponding to the task; The outstanding improvement of this embodiment compared with the prior art is: Before the asynchronous task processor sends the current node message corresponding to the task, it checks whether all the previous node messages have been successfully sent: the asynchronous task processor can check whether all the previous node messages have been successfully sent by querying the local task table: If there are previous node tasks that have not been successfully sent, pause the current task processing and trigger a delayed retry; If there are no previous node tasks that have not been successfully sent, call the MQ client to send node messages.

[0027] The logic code example of the asynchronous task processor in this embodiment is as follows: / / Code example public class AsyncTaskProcessor { / / Message topic configuration private CommonEventProvider commonEvent; / / Process tasks grouped by document number public void processTasks( ) { List <mqtask>tasks = queryPendingTasks( ); tasks.groupBy(task ->task.getOrderId( )) .forEach((orderId, taskGroup) ->{ / / Sort in node order (review → outbound → receipt → completion) sortByNodeOrder(taskGroup); for (MqTask task : taskGroup) { if (checkPreTasksCompleted(task)){ sendMsgOrderly(task,MqUtil.getDestination(commonEvent.getTopic(),commonEvent.getTag( )),true); } else { delayRetry(task); } } }); } / / Check if previous nodes are completed private boolean checkPreTasksCompleted(MqTask task) { List <string>preNodeTypes = getPreNodeTypes(task.getNodeType()); return preNodeTypes.stream( ) .allMatch(preNode -> taskDao.existsSuccessTask(task.getOrderId(),preNode)); } } / / Send sequential messages. Under the premise of node order, messages should also be sent in sequential order private void sendMsgOrderly(MqTask payload, String destination, boolean orderly) { String msgId = payload.getId( ); String hashKey = payload.getOrderId( ); RocketMQTemplate rocketMqTemplate = (RocketMQTemplate) this.rocketMqTemplateObjectProvider.getIfAvailable( ); Assert.notNull(rocketMqTemplate, "rocketMQTemplate is null"); Message<?> message = MqDtxUtil.buildRocketMqMessage(payload); log.info("send message, body: destination={},message={}", new Object[]{destination, message}); try { SendResult sendResult; if (orderly) { sendResult = rocketMqTemplate.syncSendOrderly(destination, message, hashKey, 10000L); } else { sendResult = rocketMqTemplate.syncSend(destination,message, 10000L); } if (SendStatus.SEND_OK == sendResult.getSendStatus( )) { log.info("send success,sendResult={}", sendResult); this.updateSubTaskStatus(SubMsgTaskStatus.SEND_SUCCESS); } else { log.error("sendfail,sendResult:{}", sendResult); this.updateSubTaskStatus(SubMsgTaskStatus.SEND_FAIL); } } catch (Exception var10) { log.error("Occur an Exception when innerSendMessage( )",var10); this.updateSubTaskStatus(SubMsgTaskStatus.SEND_FAIL); } } The asynchronous task processor verifies whether all the messages of the previous nodes have been successfully sent, including: 1) Determine the types of all previous nodes of the current node according to the preset node order; 2) Query the task status of the previous nodes under the same document number in the local task table. If there are unsuccessfully completed tasks, it is determined that the verification fails; Therefore, adding a composite index to the fields corresponding to the document number and node type can effectively facilitate the accelerated sequential query process here.

[0028] In this embodiment, exponential backoff strategy is adopted for delayed retry, and the retry interval is 2 n * base_delay, where n is the number of retries, n = 0, 1, 2...; base_delay is the pre-set retry time, For example, base_delay can be set to 2s, then the retry time interval satisfies: The first time is 2 0 * 2s = 2s; The second time is 2 1 * 2s = 4s; The third time is 2 2 * 2 = 8s; ...; And so on until the retry is successful, Thus, by adopting the exponential backoff strategy for delayed retry, the reliability and system load can be balanced.

[0029] If there is no previous node task that fails to be sent, when calling the MQ client to send the node message: If the sending is successful, update the task status to successful; If the sending fails, update the number of retries and trigger the retry strategy.

[0030] The triggered retry strategy is the aforementioned exponential backoff strategy; by updating the number of retries, the current number of retries can be recorded in real time. Thus, in the next execution, according to the current number of executions, the next execution time will be calculated through the formula: 2 n * base_delay. Therefore, the check of whether the previous node message is successfully sent is realized based on the document number and node order, and at the same time, the influence of the success or failure of this message sending on the delayed retry time is also considered to ensure the strict sequential sending of messages.

[0031] The system in this embodiment further includes a retry and fuse control module, and the retry and fuse control module manages the retry strategy and fuse rule of failed tasks. Specifically, in this embodiment, in the retry and fuse control module, if the number of task retries exceeds the threshold, the task is marked as failed and the fuse rule is triggered. The fuse rule includes: 1) Send a fuse warning to the user; and 2) Suspend the processing of all subsequent tasks under the same document number until manual intervention for recovery.

[0032] [Second Embodiment] The second embodiment of the present invention further provides a control method for the sequential message ordering system mentioned in the first embodiment, including the following steps: S1: The status of the service node changes, generating a corresponding node message and persistently storing it in the local task table, marking it as the to-be-sent status; S2: The asynchronous task processor groups the tasks by document number, processes the tasks in the local task table, and sends the current node message corresponding to the task; S3: Before the asynchronous task processor sends the current node message corresponding to the task, it checks whether all the previous node messages have been successfully sent; If there are previous node tasks that have not been successfully sent, suspend the current task processing and trigger a delayed retry; If there are no previous node tasks that have not been successfully sent, call the MQ client to send the node message; S4: If the sending is successful, update the task status to successful; If the sending fails, update the retry count and trigger the retry policy; S5: If the task retry count exceeds the threshold, mark the task as failed and trigger the circuit breaker rule.

[0033] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claimed rights.< / string> < / mqtask>

Claims

1. A sequential message order-preserving system, the system comprising: A business service module, which drives the task node status to flow according to the preset node sequence, and when the task node status changes, generates a corresponding node message and stores it persistently in the local task table, marking it as a pending state; A local task table, which persistently stores tasks to be sent and records the current status of tasks in real time; An asynchronous task processor, which groups tasks by document number, processes tasks in a local task table according to a preset node sequence, and sends a current node message corresponding to the task; Features: Before sending the current node message corresponding to the task, the asynchronous task processor verifies whether all previous node messages have been sent successfully: If there is a previous node task that has not been sent successfully, the current task processing will be paused and a delayed retry will be triggered; If there is no previous node task that has not been sent successfully, call the MQ client to send the node message.

2. A sequential message order preservation system according to claim 1, characterized in that: The preset node sequence is: review → delivery → receipt → completion.

3. A sequential message order-preserving system according to claim 1, characterized in that: Follow the steps below to create the local task table: 1) Create a database table; 2) Add a joint index for the fields corresponding to the document number and node type.

4. A sequential message order preservation system according to claim 1, characterized in that: The asynchronous task processor verifies whether all preceding node messages have been sent successfully by querying the local task table.

5. A sequential message order preservation system according to claim 1, characterized in that: The asynchronous task processor verifies whether all preceding node messages have been sent successfully, including: 1) According to the preset node order, determine the types of all previous nodes of the current node; 2) Query the task status of the previous node under the same document number in the local task table. If there is an unsuccessful task, it is judged as verification failure.

6. A sequential message order-preserving system according to claim 1, characterized in that: The delayed retry adopts an exponential backoff strategy with a retry interval of 2 n * base_delay, in, n is the number of retries, n=0, 1, 2...; base_delay is the preset retry time.

7. A sequential message order-preserving system according to claim 6, characterized in that: If there is no previous node task that has not been sent successfully, when calling the MQ client to send a node message: If the sending is successful, update the task status to success; If sending fails, update the retry count and trigger the retry strategy.

8. A sequential message order-preserving system according to claim 1, characterized in that: The system also includes a retry and fuse control module, which manages the retry strategy and fuse rules of failed tasks.

9. A sequential message order preservation system according to claim 8, characterized in that: In the retry and fuse control module, if the number of task retries exceeds the threshold, the task is marked as failed and the fuse rule is triggered. The fuse rule includes: 1) Send a fuse warning to the user; 2) Suspend the processing of all subsequent tasks under the same document number until they are resumed by human intervention.

10. A control method for a sequence message order-preserving system according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1: The business node status changes, the corresponding node message is generated and persistently stored in the local task table, and it is marked as pending to be sent; S2: The asynchronous task processor groups tasks by document number, processes tasks in the local task table, and sends the current node message corresponding to the task; S3: Before the asynchronous task processor sends the current node message corresponding to the task, it checks whether all the previous node messages have been sent successfully; If there is a previous node task that has not been sent successfully, the current task processing will be paused and a delayed retry will be triggered; If there is no previous node task that has not been sent successfully, call the MQ client to send the node message; S4: If the sending is successful, update the task status to success; If the sending fails, update the number of retries and trigger the retry strategy; S5: If the number of task retries exceeds the threshold, the task is marked as failed and the circuit breaker rule is triggered.

Images