A distributed transaction processing method, device and equipment

Abstract

The present application provides a distributed transaction processing method, apparatus, and device. The method includes: responding to a transaction start event, assembling a transaction message, inserting the assembled transaction message into a local transaction message table, synchronously registering a callback method, and committing a local transaction; responding to a transaction commit instruction, calling the callback method, and sending the processed transaction message to a message middleware; determining whether the transaction message sent to the message middleware was successfully sent; and processing the message based on whether the transaction message was successfully sent. The present application can solve the problem in the prior art that causes inconsistencies in transaction messages between upstream and downstream services.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of transaction processing, and more particularly to a distributed transaction processing method and apparatus. Background Art

[0002] In the prior art, there are multiple solutions for distributed transactions, including distributed transaction mechanisms and best-effort notifications based on message queues.

[0003] In a microservices architecture, two services need to communicate. There are two ways of communication between services: synchronous and asynchronous. Synchronous communication will lead to tight coupling between services, which is not conducive to high availability of the system. Asynchronous communication should be used whenever possible. When performing asynchronous communication through message middleware, due to the unreliability of the distributed system itself, it is recommended to use the best-effort notification mode. The existing best-effort notification mode cannot achieve effective repetition, which leads to inconsistent transaction messages between upstream and downstream services. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a distributed transaction processing method and apparatus. The existing best-effort notification mode cannot achieve effective repetition, which leads to inconsistencies in transaction messages between upstream and downstream services.

[0005] In a first aspect, an embodiment of the present application provides a distributed transaction processing method, the method comprising:

[0006] In response to the transaction start event, assemble the transaction message, insert the assembled transaction message into the local transaction message table of the local application, synchronously register the callback method, and commit the local transaction;

[0007] In response to a transaction commit instruction, the callback method is called to synchronously send the assembled transaction message to the message middleware;

[0008] Determine whether the message sent to the message middleware is sent successfully;

[0009] Process the transaction message according to whether the transaction message is sent successfully.

[0010] As an embodiment of the present application, the callback method is set in a one-to-one correspondence with the processed transaction message.

[0011] As an embodiment of the present application, the response before the transaction event is started includes:

[0012] Determine whether to open a local transaction. If so, determine whether there is an exception during the transaction execution. If so, roll back the transaction and the transaction message. If not, commit the transaction.

[0013] If local transactions are not enabled, an error message will be displayed.

[0014] As an embodiment of the present application, before responding to the transaction start event, the method further includes:

[0015] Start a scheduled task to resend failed messages to the message middleware.

[0016] As an embodiment of the present application, a database lock is set on the database, and an instance that first opens the database lock in the same service is obtained;

[0017] The message that failed to be sent is resent to the message middleware through the instance that first unlocks the database lock.

[0018] As an embodiment of the present application, before responding to the transaction event, the step further includes identifying whether the transaction message has a one-to-one corresponding identifier;

[0019] If not, a one-to-one corresponding identifier is generated for the transaction message.

[0020] As an embodiment of the present application, processing the transaction message according to whether the transaction message is sent successfully includes:

[0021] Updating the transaction message status;

[0022] The transaction message is processed according to the transaction message status.

[0023] As an embodiment of the present application, processing the transaction message according to the transaction message status includes:

[0024] If the transaction message is sent successfully, deleting the successfully sent message;

[0025] If the transaction message fails to be sent, the failed message will be marked and will be resent after subsequent task scheduling.

[0026] As an embodiment of the present application, if the message fails to be sent, the failed message will be marked and resent after subsequent task scheduling, including:

[0027] Check whether there are any messages that failed to be sent;

[0028] If yes, obtain the time log of the failed sending messages, and determine the order in which each failed sending message was sent according to the time log;

[0029] The failed messages are sent to the message middleware according to the order in which they are sent.

[0030] As an embodiment of the present application, it also includes:

[0031] The message sent to the consumer end through the message middleware is processed by the consumer end according to the unique identifier of the message.

[0032] In a second aspect, an embodiment of the present application further provides a transaction processing device, comprising:

[0033] The first processing module is configured to, in response to a transaction start event, assemble a transaction message, insert the assembled transaction message into a local transaction message table of a local application, synchronously register a callback method, and commit the local transaction;

[0034] A sending module, configured to respond to a transaction commit instruction, call the callback method, and synchronously send the assembled transaction message to the message middleware;

[0035] A determination module, used to determine whether the message sent to the message middleware is sent successfully;

[0036] The second processing module is configured to process the transaction message according to whether the transaction message is sent successfully.

[0037] In a third aspect, an embodiment of the present application further provides a transaction processing device, including:

[0038] a processor, a memory, and a computer program stored on the memory and executable on the processor;

[0039] Wherein, the processor implements the above-mentioned method when running the computer program.

[0040] Compared with the prior art, the present application provides a distributed transaction processing method and device,

[0041] In response to a transaction start event, a transaction message is assembled and inserted into the local transaction message table of the business application party, a callback method is synchronously registered, and a local transaction is committed; in response to a transaction commit instruction, the callback method is called and the assembled transaction message is synchronously sent to the message middleware; it is determined whether the message sent to the message middleware is sent successfully; and the transaction message is processed according to whether the transaction message is sent successfully. This application achieves effective repeated sending through a best-effort notification mode, so that transaction messages between upstream and downstream services will not be inconsistent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Non-limiting and non-exhaustive embodiments of the present invention are described, by way of example, with reference to the following figures, in which:

[0043] Figure 1A flowchart of a transaction processing method according to the first embodiment of the present application is shown;

[0044] Figure 2 A flowchart of a transaction processing method according to the second embodiment of the present application is shown;

[0045] Figure 3 A schematic diagram of the structure of a transaction processing device according to the first embodiment of the present application is shown;

[0046] Figure 4 A structural diagram of a transaction processing device according to embodiment 1 of the present application is shown. DETAILED DESCRIPTION

[0047] In order to make the above and other features and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.

[0048] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0049] In the embodiment of the present application, the upstream service is the service where the producer that sends messages to the message middleware is located; the downstream service is the service where the consumer that monitors and receives the message middleware messages is located.

[0050] The first embodiment of the present application relates to a distributed transaction processing method. Figure 1 A flow chart of the distributed processing method is shown in FIG. Figure 1 As shown, the distributed transaction processing method includes the following steps:

[0051] In step S101, in response to a transaction start event, a transaction message is assembled, and the assembled transaction message is inserted into the local transaction message table of the local application, a callback method is synchronously registered, and the local transaction is committed;

[0052] In an embodiment of the present application, it is determined whether the business caller is a business caller and whether a transaction is opened. If the transaction is not opened, an error is reported; if so, a transaction message is assembled and the assembled transaction message is inserted into the local transaction message table of the local application party to open a local transaction. Within the transaction, conventional business operations and two things of the present application are performed: temporarily storing transaction message information and registering a callback method with the spring synchronous transaction manager. The callback method is similar to a hook and will not be called until the transaction is committed. The message used in the callback method and the temporarily stored transaction message are the same object. In the callback method, a thread is created to synchronously send transaction messages and is placed in the thread pool. The actual situation is that the actions in the callback method are only called when the local transaction is committed. Among them, the callback method is set in a one-to-one correspondence with the assembled message, that is, when the message is obtained, the corresponding callback method is registered.

[0053] Before responding to the transaction start event, it includes determining whether to start a local transaction. If the local transaction is started, and during the transaction execution, it is determined whether there is an exception. If there is an exception, the transaction is rolled back, and the messages in the transaction message list are also rolled back; if there is no exception, the transaction is committed; if the local transaction is not started, an error reminder is issued.

[0054] A scheduled task is started with the application to obtain messages that failed to be sent from the database where the application is located and resend them to the message middleware.

[0055] In step S102, in response to the transaction commit instruction, the callback method is called to send the assembled transaction message to the message middleware in a synchronous manner;

[0056] For example, the transaction message sending thread and the internal logic of the transaction message sending thread are placed in the thread pool and: the assembled transaction message is sent synchronously, and the transaction message status is processed; in the callback method, the transaction message sending thread sends the message by synchronously sending the message. The synchronous sending method has high reliability. Either the message is sent successfully or an exception occurs. If an exception occurs, it can be captured and processed accordingly, and it will not directly cause the loss of the message like the "fire and forget" method. However, the performance of the synchronous sending method will be much worse, and it is necessary to block and wait for one message to be sent before sending the next one. However, this application uses a thread pool for reuse to avoid the performance consumption of frequent thread creation and destruction.

[0057] For example, the present application uses a spring transaction hook. After the transaction is committed, a synchronous callback method is used to send a message. Here, a thread pool is used to send messages to avoid synchronizing callback events with business execution, causing normal business waiting and blocking the main thread. After sending the message, it will monitor whether the message is sent successfully. In the embodiment of the present application, 5 core threads are used to send messages during initialization. The number of threads mentioned above can be adjusted and configured. The thread pool is a spring Bean object that is initialized when the application starts and destroyed when the application is closed. The message status will be marked whether the message is sent successfully or failed. The embodiment of the present application does not rely on the message middleware. If there is a problem with the message middleware, it is also allowed, ensuring the high availability of the current service. The message sending thread uses a synchronous sending method to improve reliability when sending messages. Either the message is sent successfully or an exception occurs. If an exception occurs, the exception can be caught and the message marked as an exception. However, the performance of synchronous sending will be much worse. The present application uses a thread pool to ensure thread reuse and avoid the performance overhead of frequent thread creation and destruction.

[0058] In step S103, it is determined whether the transaction message sent to the message middleware is sent successfully;

[0059] In step S104, the message is processed according to whether the message is sent successfully.

[0060] Specifically including: updating the status of the transaction message list;

[0061] The message is processed according to the status of the message list.

[0062] Exemplarily, updating the status of the transaction message list includes, if the message is sent successfully, updating the status of the successfully sent message in the message list from an unsent state to a successfully sent state;

[0063] If the message is not successfully sent to the message middleware, the status of the failed message in the message list is updated from the unsent state to the failed sending state.

[0064] Process the message according to the status of the message list, including: if the message is sent successfully, deleting the successfully sent message in the message list; if the message fails to be sent, marking the failed message and resending it after subsequent task scheduling until it is successfully sent to the message middleware.

[0065] This application uses the best-effort notification mode to achieve effective repeated sending, so that there will be no inconsistency in transaction messages between upstream and downstream services. That is to say, all messages to be sent in the message list will be successfully sent to the consumer end. In this application, all details of steps S101-S104 can be assembled in a starter component.

[0066] refer to Figure 2 , the specific steps of this method are as follows:

[0067] Step 201, start a local transaction;

[0068] In the above steps, it is determined whether a local transaction is started. If a local transaction is started, and during the transaction execution, it is determined whether there is an exception. If there is an exception, the transaction is rolled back and the messages related to the transaction are also rolled back. If there is no exception, the transaction is committed.

[0069] A local transaction is started, and within it, regular business operations are performed, along with two tasks for this application: temporarily storing transaction message information and registering a callback method with the Spring synchronous transaction manager. The callback method, like a hook, is invoked only after the transaction is committed. The message used in the callback method and the temporarily stored transaction message are the same object. In the callback method, a thread is created to synchronously send the transaction message and placed in the thread pool. In reality, the actions in the callback method are only invoked when the local transaction is committed.

[0070] If local transactions are not enabled, an error message will be displayed.

[0071] Step 202: Identify whether the transaction message has a one-to-one corresponding identifier. If not, generate a one-to-one corresponding identifier for the transaction message.

[0072] For example, local transaction message records have their own unique id and messageId. The unique id ensures that there is no conflict in the business database. When the message is initialized, it is first determined whether the consumer has passed the messageId. If not, a messageId is generated by default. The messageId is generated using UUID by default. This application can ensure that the message sent to the message middleware has a unique messageId, so that downstream services can perform idempotent processing.

[0073] Step S203, start the class scheduled task;

[0074] Exemplarily, a scheduled task is started when the application is started, and is used to resend messages that have failed to be sent to the message middleware.

[0075] Step S204: In response to the transaction start event, assemble a transaction message, and insert the assembled transaction message into the local transaction message table of the local application.

[0076] Step S205: In response to the transaction commit instruction, the callback method is called to send the assembled transaction message to the message middleware in a synchronous manner;

[0077] For example, assembling a message is to adjust the format of the message, and the transaction commit instruction is to try to send the transaction message to the message middleware as soon as possible, avoiding waiting until a scheduled task scans the pending messages before sending the transaction message, thereby ensuring the real-time sending of the transaction message.

[0078] Step S206, determining whether the transaction message is successfully sent to the message middleware;

[0079] Step S207, updating the message status in the message list;

[0080] For example, if the message is sent successfully, the transaction message status is updated from the pending state to the successfully sent state;

[0081] If the message is not successfully sent to the message middleware, the message status of the transaction message that failed to be sent is updated from the waiting state to the failed sending state.

[0082] Step S208: resend the message that failed to be sent to the message middleware.

[0083] Exemplarily, a time log of failed sending messages is obtained, and the order in which each failed sending message is sent is determined according to the time log;

[0084] The failed messages are sent to the message middleware according to the order in which they are sent.

[0085] Step S209: Determine whether the message resent to the message middleware is sent successfully.

[0086] In step S210, if the message is sent successfully, the successfully sent message in the message list is deleted and step S210 is executed; if the message is sent unsuccessfully, the failed message is marked and resent after subsequent task scheduling until it is successfully sent to the message middleware.

[0087] Step S211: Send the message to the consumer end through the message middleware for consumption.

[0088] In step S212, the consumer performs idempotence processing according to the unique identifier of the message.

[0089] For example, the consumer side needs to enable the ack confirmation mechanism of the message to avoid message loss due to middleware problems. The consumer side needs to implement idempotent processing to avoid repeated consumption problems caused by the upstream guaranteeing that the message is to be sent, being sent to the message middleware multiple times, and being consumed multiple times by the downstream.

[0090] When the embodiment of the present application performs asynchronous communication through the message middleware, the transaction messages between the upstream and downstream services remain consistent. When problems occur in the message middleware, the upstream service can be fault-tolerant and used normally. Only messages that the upstream service is sure to send to the downstream are sent to the message middleware, and it is guaranteed that they will eventually be sent to the message middleware. Even if problems occur in the message middleware, it will not be affected. At the same time, the message is marked with a unique mark, which is conducive to idempotent processing of the downstream service.

[0091] In the embodiment of the present application, when the message middleware on which it depends is unavailable, the sending of messages is not affected. When the message middleware is available, the message will continue to be pushed. Upstream service developers do not need to consider the order in which the message is sent to the message middleware and other business codes are written, avoiding the problem of considering the order of sending the message to the message middleware code and the business code when writing the code. The upstream service will make its best effort to send the message to be sent to the message middleware so that the downstream service can consume it; similar to the scheduled task, the message record in the local transaction table of successful sending is deleted regularly to avoid unlimited expansion of the local transaction message table; based on the database lock, it is ensured that when the scheduled task is executed in multiple instances of the same service, only one of the service instances can execute it.

[0092] Second, as Figure 3 As shown, the embodiment of the present application further provides a transaction processing device, including:

[0093] The first processing module 31 is used to assemble a transaction message in response to a transaction start event, insert the assembled transaction message into the local transaction message table of the local application, synchronously register a callback method, and commit the local transaction;

[0094] The sending module 32 is used to call the callback method in response to the transaction commit instruction and send the assembled transaction message to the message middleware in a synchronous manner;

[0095] A determination module 33, configured to determine whether the message sent to the message middleware is sent successfully;

[0096] The second processing module 34 is configured to process the transaction message according to whether the transaction message is sent successfully.

[0097] The device provided in the embodiment of the present application can be specifically used to execute the method embodiment provided in the above-mentioned embodiment 2, which will not be described in detail here.

[0098] This application uses the best-effort notification mode to achieve effective repeated sending, so that there will be no inconsistency in transaction messages between upstream and downstream services. That is to say, all messages to be sent in the message list will be successfully sent to the consumer end. In this application, all details of steps S101-S104 can be assembled in a starter component.

[0099] Thirdly, as Figure 4 As shown, the present application also provides a transaction processing device, including:

[0100] A processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401;

[0101] Wherein, when the processor 401 runs the computer program, the method described in the first aspect is implemented.

[0102] The beneficial effects of a transaction processing device provided by an embodiment of the present application are the same as those of the above-mentioned embodiments and will not be repeated here.

[0103] At least one technical feature of the above embodiments can be arbitrarily combined. In order to make the description concise, not all possible combinations of at least one technical feature in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0104] Although the present invention has been described in conjunction with the embodiments, it will be appreciated by those skilled in the art that the above description and the accompanying drawings are exemplary and non-restrictive, and the present invention is not limited to the disclosed embodiments. Various modifications and variations are possible without departing from the spirit of the present invention.

Claims

1. A distributed transaction processing method, characterized in that: The method comprises, In response to the transaction start event, assemble the transaction message, insert the assembled transaction message into the local transaction message table of the local application, synchronously register the callback method, and commit the local transaction; In response to the transaction commit instruction, calling the callback method, and sending the assembled transaction message to the message middleware in a synchronous manner; Determine whether the message sent to the message middleware is sent successfully; Processing the transaction message according to whether the transaction message is sent successfully; The callback method is set in one-to-one correspondence with the assembly transaction message; The response before the transaction event is started includes: Determine whether to open a local transaction. If so, determine whether there is an exception during the transaction execution. If so, roll back the transaction and the transaction message. If there is no exception, the transaction is committed; If local transactions are not enabled, an error message will be displayed. and / or The response before the transaction event is started includes: Start a scheduled task to obtain failed messages from the database where the application is located and resend them to the message middleware; and / or The response before the transaction event is started includes: Identify whether the transaction message has a one-to-one corresponding identifier; If not, a one-to-one corresponding identifier is generated for the transaction message.

2. A distributed transaction processing method according to claim 1, characterized in that: A database lock is set on the database, and the instance that first opens the database lock in the same service is obtained; The message that failed to be sent is resent to the message middleware through the instance that first unlocks the database lock.

3. A distributed transaction processing method according to claim 1, characterized in that: The processing of the transaction message according to whether the transaction message is sent successfully includes: Updating the transaction message status; The transaction message is processed according to the transaction message status.

4. A distributed transaction processing method according to claim 3, characterized in that: said processing of the transaction message according to the transaction message state, include, If the transaction message is sent successfully, deleting the successfully sent message; If the transaction message fails to be sent, the failed message will be marked and will be resent after subsequent task scheduling.

5. A distributed transaction processing method according to claim 4, characterized in that: If the message fails to be sent, the failed message will be marked and resent after subsequent task scheduling, including: Check whether there are any messages that failed to be sent; If yes, obtain the time log of the failed sending messages, and determine the order in which each failed sending message was sent according to the time log; The failed messages are sent to the message middleware according to the order in which they are sent.

6. A distributed transaction processing method according to claim 1, characterized in that: Also includes, The message sent to the consumer end through the message middleware is processed by the consumer end according to the unique identifier of the message.

7. A transaction processing device, characterized in that: include, The first processing module is configured to, in response to a transaction start event, assemble a transaction message, insert the assembled transaction message into a local transaction message table of a local application, synchronously register a callback method, and commit the local transaction; A sending module is used to call the callback method in response to the transaction commit instruction and send the assembled transaction message to the message middleware in a synchronous manner; A determination module, used to determine whether the message sent to the message middleware is sent successfully; A second processing module, configured to process the transaction message according to whether the transaction message is sent successfully; The callback method is set in one-to-one correspondence with the assembly transaction message; The response before the transaction event is started includes: Determine whether to open a local transaction. If so, determine whether there is an exception during the transaction execution. If so, roll back the transaction and the transaction message. If there is no exception, the transaction is committed; If local transactions are not enabled, an error message will be displayed. and / or The response before the transaction event is started includes: Start a scheduled task to obtain failed messages from the database where the application is located and resend them to the message middleware; and / or The response before the transaction event is started includes: Identify whether the transaction message has a one-to-one corresponding identifier; If not, a one-to-one corresponding identifier is generated for the transaction message.

8. A transaction processing device, characterized in that: include, a processor, a memory, and a computer program stored on the memory and executable on the processor; Wherein, when the processor runs the computer program, the method according to any one of claims 1 to 6 is implemented.

Images